AI Coding Newsletter

AI Coding Newsletter — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.

  • Inductive bias

    Inductive bias

    The inductive bias (also known as learning bias) of a learning algorithm is the set of assumptions that the learner uses to predict outputs of given inputs that it has not encountered. Inductive bias is anything which makes the algorithm learn one pattern instead of another pattern (e.g., step-functions in decision trees instead of continuous functions in linear regression models). Learning involves searching a space of solutions for a solution that provides a good explanation of the data. However, in many cases, there may be multiple equally appropriate solutions. An inductive bias allows a learning algorithm to prioritize one solution (or interpretation) over another, independently of the observed data. In machine learning, the aim is to construct algorithms that are able to learn to predict a certain target output. To achieve this, the learning algorithm is presented some training examples that demonstrate the intended relation of input and output values. Then the learner is supposed to approximate the correct output, even for examples that have not been shown during training. Without any additional assumptions, this problem cannot be solved since unseen situations might have an arbitrary output value. The kind of necessary assumptions about the nature of the target function are subsumed in the phrase inductive bias. A classical example of an inductive bias is Occam's razor, assuming that the simplest consistent hypothesis about the target function is actually the best. Here, consistent means that the hypothesis of the learner yields correct outputs for all of the examples that have been given to the algorithm. Approaches to a more formal definition of inductive bias are based on mathematical logic. Here, the inductive bias is a logical formula that, together with the training data, logically entails the hypothesis generated by the learner. However, this strict formalism fails in many practical cases in which the inductive bias can only be given as a rough description (e.g., in the case of artificial neural networks), or not at all. == Types == The following is a list of common inductive biases in machine learning algorithms. Maximum conditional independence: if the hypothesis can be cast in a Bayesian framework, try to maximize conditional independence. This is the bias used in the Naive Bayes classifier. Minimum cross-validation error: when trying to choose among hypotheses, select the hypothesis with the lowest cross-validation error. Although cross-validation may seem to be free of bias, the "no free lunch" theorems show that cross-validation must be biased, for example assuming that there is no information encoded in the ordering of the data. Maximum margin: when drawing a boundary between two classes, attempt to maximize the width of the boundary. This is the bias used in support vector machines. The assumption is that distinct classes tend to be separated by wide boundaries. Minimum description length: when forming a hypothesis, attempt to minimize the length of the description of the hypothesis. Minimum features: unless there is good evidence that a feature is useful, it should be deleted. This is the assumption behind feature selection algorithms. Nearest neighbors: assume that most of the cases in a small neighborhood in feature space belong to the same class. Given a case for which the class is unknown, guess that it belongs to the same class as the majority in its immediate neighborhood. This is the bias used in the k-nearest neighbors algorithm. The assumption is that cases that are near each other tend to belong to the same class. == Shift of bias == Although most learning algorithms have a static bias, some algorithms are designed to shift their bias as they acquire more data. This does not avoid bias, since the bias shifting process itself must have a bias.

    Read more →
  • Artificial intelligence industry in Taiwan

    Artificial intelligence industry in Taiwan

    The artificial intelligence (AI) industry in Taiwan refers to the development, application, and commercialization of artificial intelligence technologies within Taiwan. The industry has grown alongside Taiwan's established strengths in semiconductor manufacturing and information and communications technology (ICT), and is supported by government policy, research institutions, and private sector participation. AI development in Taiwan has focused on integrating hardware capabilities with software applications across sectors such as manufacturing, healthcare, and smart infrastructure. Artificial intelligence has been identified as a strategic area of development in Taiwan since the late 2010s. While Taiwan has historically played a limited role in early theoretical and expert-system phases of AI development, its position in global electronics manufacturing has provided a foundation for participation in the contemporary era of machine learning and data-driven AI systems. Taiwan's AI industry is characterized by a strong hardware base, particularly in semiconductor production and AI server manufacturing, combined with increasing investment in software, data infrastructure, and applied AI services. The sector has been shaped by global demand for computing power, advances in deep learning, and the expansion of AI applications in industrial and commercial contexts. == Government policy and development == The Taiwanese government has promoted AI development through a series of national strategies. In 2017, the Ministry of Science and Technology launched the "AI Grand Strategy for a Small Country" initiative, investing approximately US$517 million between 2017 and 2021 to support research, infrastructure, and talent development. This initiative aimed to build a domestic AI ecosystem by funding research centers, expanding data infrastructure, and supporting industrial adoption. The Executive Yuan also introduced the AI Taiwan Action Plan 1.0 (2018–2021), which focused on integrating AI technologies into existing industries and strengthening research and development capabilities. A subsequent plan, AI Taiwan Action Plan 2.0 (2023–2026), expanded the focus to include ethical governance, regulatory frameworks, and risk management in response to the growth of generative AI technologies. In 2023, the Taiwan AI Center of Excellence (Taiwan AICoE), a government-backed hub, was established by the National Science and Technology Council to accelerate AI development, foster international collaboration, and train talent in Taiwan. It acts as a specialized think tank focusing on creating a "smart technology island" by integrating AI resources and developing trusted, human-centric AI technologies. In 2024, the Taiwan Chip-based Industrial Innovation Program (CbI) was launched by the Executive Yuan as a 10-year, NT$300 billion (US$9.3 billion) initiative to leverage Taiwan's semiconductor dominance, driving innovation in AI, smart mobility, manufacturing, and healthcare. It aims to combine generative AI with IC technology, cultivate talent, and attract global startups to build a "Silicon Island". In parallel, the Taiwanese government has explored legislative frameworks such as a proposed Artificial Intelligence Fundamental Act in December 2025, addressing issues including data protection, safety standards, and intellectual property. == Industrial structure == === Semiconductor and hardware foundation === Taiwan's AI industry is closely linked to its semiconductor sector. In 2020, Taiwan accounted for approximately 77.3% of the global wafer foundry market and 57.7% of packaging and testing, with a 20.1% share in integrated circuit (IC) design. These capabilities provide critical infrastructure for AI systems, which rely on high-performance computing hardware. Taiwanese firms are also involved in the production of AI servers and related components, contributing significantly to global supply chains for data centers and cloud computing. The integration of chip design, manufacturing, and assembly has enabled Taiwan to play a central role in providing the computational resources required for AI development. On 20 November 2025, Google established the "Google Taiwan AI Infrastructure R&D Center", second only to its US headquarters and largest AI hardware infrastructure engineering center outside of the United States. === Software and services === Compared to its hardware capabilities, Taiwan's AI software sector is less developed. The absence of large-scale global AI platform companies has been noted as a structural limitation. As a result, much of Taiwan's AI industry focuses on applied solutions, including customization of existing AI models for specific industries. Therefore, efforts to strengthen software capabilities have included investment in research institutions, startup ecosystems, and collaborations between academia and industry. == Applications == === Smart manufacturing === AI has been widely applied in Taiwan's manufacturing sector, which is a major component of the economy. Applications include process automation, predictive maintenance, quality control, and fault detection. AI-enabled smart manufacturing systems aim to improve efficiency, reduce production costs, and enhance product quality. Taiwan's manufacturing industry has incorporated AI technologies into production lines, particularly in electronics and machinery sectors. === Healthcare === The use of AI in healthcare in Taiwan has expanded in areas such as medical imaging, diagnostics, and drug development. AI systems are used to analyze CT scans, MRI data, and other clinical information to support diagnosis and treatment planning. Taiwan's healthcare sector, which includes medical devices, pharmaceuticals, and medical services, has benefited from the integration of AI technologies, particularly in precision medicine and clinical decision support systems. A notable example of AI healthcare deployment in Taiwan is the collaboration between Siemens Healthineers, Ever Fortune AI, and Asia University Hospital. === Edge computing and IoT === AI applications in Taiwan increasingly involve edge computing, where data processing occurs near the source rather than in centralized cloud systems. This approach reduces latency and bandwidth requirements and is used in smart devices, sensors, and industrial equipment. Edge AI technologies are applied in areas such as smart appliances, industrial automation, and transportation systems. == Education and talent development == Human capital development has been a key focus of Taiwan's AI strategy. The Taiwan AI Academy, established in 2018 with support from Academia Sinica and industry partners, provides training programs for professionals and students aimed at accelerating the adoption of artificial intelligence technologies across industries. The academy offers a range of courses, including executive-level programs, technical training, and specialized tracks in areas such as smart manufacturing, smart healthcare, and edge AI. These programs are designed to provide intensive and practical instruction over relatively short periods. A notable component of the curriculum is project-based learning, in which participants are required to complete proof-of-concept (POC) projects addressing real-world industrial problems. These projects are often developed further for implementation within companies, facilitating technology transfer and commercialization. Between 2018 and 2021, more than 8,000 individuals completed AI training programs across campuses in Taipei, Hsinchu, Taichung, and Tainan. Graduates of the academy have contributed to the introduction of AI systems in sectors such as manufacturing, healthcare, and finance, supporting broader industrial transformation efforts. In addition to the Taiwan AI Academy, universities and research institutions in Taiwan play a significant role in AI education and research. Leading universities have expanded programs in computer science, data science, and machine learning, while research institutes conduct applied and fundamental studies in artificial intelligence. Collaboration between academia, government, and industry is a common feature of Taiwan's AI ecosystem, with joint research projects, internship programs, and technology incubation initiatives supporting talent development. Government-supported initiatives have also sought to attract and retain AI talent, including funding for graduate education, international collaboration programs, and incentives for industry–academic partnerships. These efforts aim to address talent shortages and strengthen Taiwan's capacity in both applied and foundational AI research. == Regulation and governance == Taiwan has developed guidelines and policy frameworks to address the risks associated with AI technologies. In 2023, the Executive Yuan issued guidelines for the use of generative AI in government agencies, focusing on data security and privacy. Ongoing policy discussions hav

    Read more →
  • Magic state distillation

    Magic state distillation

    Magic state distillation is a method for creating more accurate quantum states from multiple noisy ones, which is important for building fault tolerant quantum computers. It has also been linked to quantum contextuality, a concept thought to contribute to quantum computers' power. The technique was first proposed by Emanuel Knill in 2004, and further analyzed by Sergey Bravyi and Alexei Kitaev the same year. Thanks to the Gottesman–Knill theorem, it is known that some quantum operations (operations in the Clifford group) can be perfectly simulated in polynomial time on a classical computer. In order to achieve universal quantum computation, a quantum computer must be able to perform operations outside this set. Magic state distillation achieves this, in principle, by concentrating the usefulness of imperfect resources, represented by mixed states, into states that are conducive for performing operations that are difficult to simulate classically. A variety of qubit magic state distillation routines and distillation routines for qubits with various advantages have been proposed. == Stabilizer formalism == The Clifford group consists of a set of n {\displaystyle n} -qubit operations generated by the gates {H, S, CNOT} (where H is Hadamard and S is [ 1 0 0 i ] {\displaystyle {\begin{bmatrix}1&0\\0&i\end{bmatrix}}} ) called Clifford gates. The Clifford group generates stabilizer states which can be efficiently simulated classically, as shown by the Gottesman–Knill theorem. This set of gates with a non-Clifford operation is universal for quantum computation. == Magic states == Magic states are purified from n {\displaystyle n} copies of a mixed state ρ {\displaystyle \rho } . These states are typically provided via an ancilla to the circuit. A magic state for the π / 6 {\displaystyle \pi /6} rotation operator is | M ⟩ = cos ⁡ ( β / 2 ) | 0 ⟩ + e i π 4 sin ⁡ ( β / 2 ) | 1 ⟩ {\displaystyle |M\rangle =\cos(\beta /2)|0\rangle +e^{i{\frac {\pi }{4}}}\sin(\beta /2)|1\rangle } where β = arccos ⁡ ( 1 3 ) {\displaystyle \beta =\arccos \left({\frac {1}{\sqrt {3}}}\right)} . A non-Clifford gate can be generated by combining (copies of) magic states with Clifford gates. Since a set of Clifford gates combined with a non-Clifford gate is universal for quantum computation, magic states combined with Clifford gates are also universal. == Purification algorithm for distilling |M〉 == The first magic state distillation algorithm, invented by Sergey Bravyi and Alexei Kitaev, is as follows. Input: Prepare 5 imperfect states. Output: An almost pure state having a small error probability. repeat Apply the decoding operation of the five-qubit error correcting code and measure the syndrome. If the measured syndrome is | 0000 ⟩ {\displaystyle |0000\rangle } , the distillation attempt is successful. else Get rid of the resulting state and restart the algorithm. until The states have been distilled to the desired purity.

    Read more →
  • OpenWSN

    OpenWSN

    OpenWSN aims to build an open standard-based and open source implementation of a complete constrained network protocol stack for wireless sensor networks and Internet of Things. The project was created at the University of California Berkeley and extended at the INRIA and at the Open University of Catalonia (UOC). The root of OpenWSN is a deterministic MAC layer implementing the IEEE 802.15.4e TSCH based on the concept of Time Slotted Channel Hopping (TSCH). Above the MAC layer, the Low Power Lossy Network stack is based on IETF standards including the IETF 6TiSCH management and adaptation layer (a minimal configuration profile, 6top protocol and different scheduling functions). The stack is complemented by an implementation of 6LoWPAN, RPL in non-storing mode, UDP and CoAP, enabling access to devices running the stack from the native IPv6 through open standards. OpenWSN is related to other projects including the following: RIOT OpenMote OpenWSN is available for Linux, Windows and OS X platforms. Current release of OpenWSN is 1.14.0.

    Read more →
  • Security switch

    Security switch

    A security switch is a hardware device designed to protect computers, laptops, smartphones and similar devices from unauthorized access or operation, distinct from a virtual security switch which offers software protection. Security switches should be operated by an authorized user only; for this reason, it should be isolated from other devices, in order to prevent unauthorized access, and it should not be possible to bypass it, in order to prevent malicious manipulation. The primary purpose of a security switch is to provide protection against surveillance, eavesdropping, malware, spyware, and theft of digital devices. Unlike other protections or techniques, a security switch can provide protection even if security has already been breached, since it does not have any access from other components and is not accessible by software. It can additionally disconnect or block peripheral devices, and perform "man in the middle" operations. A security switch can be used for human presence detection since it can only be initiated by a human operator. It can also be used as a firewall. == Types == === Hardware kill switch === A hardware kill switch (HKS) is a physical switch that cuts the signal or power line to the device or disable the chip running them. == Examples == A cellphone is compromised by malicious software, and the device initiates video and audio recording. When the user activates the “prevent capture of audio/video” mode of the security switch, that either physically disconnects or cut the power to the microphone and the camera, which stops the recording. A laptop that has an embedded security switch is stolen. The security switch detects a lack of communication from a specific external source for 12 hours, and responds by disconnecting the screen, keyboard and other key components, rendering the laptop useless, with no possibility of recovery, even with a full format. A user wishes to prevent tracking of their location. The user then activates geolocation protection and the security switch disables all GPS communication, eliminating the possibility of tracking the device's location. A user desires to eliminate the possibility of their PIN being copied from their smartphone. They can activate the secure input function, causing the security switch to disconnect the touch screen from the operating system, so input signals are not available to any devices except the switch. A security switch performs scheduled monitoring and finds that a program is attempting to download malicious content from the internet. It then activates internet security function and disables internet access, interrupting the download. If laptop software is compromised by air-gap malware, the user may activate the security switch and disconnect the speaker and microphone, so it can not establish communication with the device. == History == Google started to work on a hardware kill switch for AI in 2016. In 2019, Apple, and Google, along with a handful of smaller players, are designing “kill switches” that cut the power to the microphones or cameras in their devices. Googles first product that implemented this is Nest Hub Max. Hardware kill switches are already available and widely tested on the PinePhone, Librem, Shiftphone, to cut power to the input peripherals (microphone, camera) but also the network connectivity modules (wifi, cellular network).

    Read more →
  • DIKW pyramid

    DIKW pyramid

    The DIKW pyramid (also known as the knowledge pyramid or information hierarchy) is a model describing relationships between data, information, knowledge and wisdom sometimes also stylized as a chain, refer to models of possible structural and functional relationships between a set of components—often four, data, information, knowledge, and wisdom. The concept has roots predating the 1980s. In the latter years of that decade, interest in the models grew after explicit presentations and discussions, including from Milan Zeleny, Russell Ackoff, and Robert W. Lucky. Subsequent important discussions extended along theoretical and practical lines into the coming decades. While debate continues as to actual meaning of the component terms of DIKW-type models, and the actual nature of their relationships—including occasional doubt being cast over any simple, linear, unidirectional model—even so they have become very popular visual representations in use by business, the military, and others. Among the academic and popular, not all versions of the DIKW-type models include all four components (earlier ones excluding data, later ones excluding or downplaying wisdom, and several including additional components (for instance Ackoff inserting "understanding" before and Zeleny adding "enlightenment" after the wisdom component). In addition, DIKW-type models are no longer always presented as pyramids, instead also as a chart or framework (e.g., by Zeleny), as flow diagrams (e.g., by Liew, and by Chisholm et al.), and sometimes as a continuum (e.g., by Choo et al.). == Short description == As Rowley noted in 2007, the DIKW model "is often quoted, or used implicitly, in definitions of data, information and knowledge in the information management, information systems and knowledge management literatures, but [as of that date] there ha[d] been limited direct discussion of the hierarchy". Reviews of textbooks and a survey of scholars in relevant fields indicate that there was not a consensus as to definitions used in the model as of that date, and as reviewed by Liew in that year, even less "in the description of the processes that transform components lower in the hierarchy into those above them". Zins work, published in 2007—from studies in 2003-2005 that documented "130 definitions of data, information, and knowledge formulated by 45 scholars", published in 2007—to suggest that the data–information–knowledge components of DIKW refer to a class of no less than five models, as a function of whether data, information, and knowledge are each conceived of as subjective, objective (what Zins terms, "universal" or "collective") or both. In Zins' usage, subjective and objective "are not related to arbitrariness and truthfulness, which are usually attached to the concepts of subjective knowledge and objective knowledge". Information science, Zins argues, studies data and information, but not knowledge, as knowledge is an internal (subjective) rather than an external (universal–collective) phenomenon. == Representations == === Graphical representation === DIKW is a hierarchical model often depicted as a pyramid, sometimes as a chain, with data at its base and wisdom at its apex (or chain-beginning and -end). Both Zeleny and Ackoff have been credited with originating the pyramid representation, although neither used a pyramid to present their ideas. According to Wallace, Debons and colleagues may have been the first to "present the hierarchy graphically". Many variations of the DIKW-type pyramid have been produced. One, in use by knowledge managers in the United States Department of Defense, attempts to show the DIKW progression to enable effective decisions and consequent activities supporting shared understanding throughout defense organizations, as well as supporting management of risks associated with decisions. DIKW-type hierarchical information paradigms have also been represented as two-dimensional charts, and as flow diagrams, where relationships between the components may be presented less hierarchically, with defining aspects of the relationships, feedback loops, etc. === Computational representation === Intelligent decision support systems are trying to improve decision making by introducing new technologies and methods from the domain of modeling and simulation in general, and in particular from the domain of intelligent software agents in the contexts of agent-based modeling. The following example describes a military decision support system, but the architecture and underlying conceptual idea are transferable to other application domains: The value chain starts with data quality describing the information within the underlying command and control systems. Information quality tracks the completeness, correctness, currency, consistency and precision of the data items and information statements available. Knowledge quality deals with procedural knowledge and information embedded in the command and control system such as templates for adversary forces, assumptions about entities such as ranges and weapons, and doctrinal assumptions, often coded as rules. Awareness quality measures the degree of using the information and knowledge embedded within the command and control system. Awareness is explicitly placed in the cognitive domain. By the introduction of a common operational picture, data are put into context, which leads to information instead of data. The next step, which is enabled by service-oriented web-based infrastructures (but not yet operationally used), is the use of models and simulations for decision support. Simulation systems are the prototype for procedural knowledge, which is the basis for knowledge quality. Finally, using intelligent software agents to continually observe the battle sphere, apply models and simulations to analyze what is going on, to monitor the execution of a plan, and to do all the tasks necessary to make the decision maker aware of what is going on, command and control systems could even support situational awareness, the level in the value chain traditionally limited to pure cognitive methods. == History == Danny P. Wallace, a professor of library and information science, explained that the origin of the DIKW pyramid is uncertain: The presentation of the relationships among data, information, knowledge, and sometimes wisdom in a hierarchical arrangement has been part of the language of information science for many years. Although it is uncertain when and by whom those relationships were first presented, the ubiquity of the notion of a hierarchy is embedded in the use of the acronym DIKW as a shorthand representation for the data-to-information-to-knowledge-to-wisdom transformation.Many authors think that the idea of the DIKW relationship originated from two lines in the poem "Choruses", by T. S. Eliot, that appeared in the pageant play The Rock, in 1934: === Knowledge, intelligence, and wisdom === In 1927, Clarence W. Barron addressed his employees at Dow Jones & Company on the hierarchy: "Knowledge, Intelligence and Wisdom". === Data, information, knowledge === In 1955, English-American economist and educator Kenneth Boulding presented a variation on the hierarchy consisting of "signals, messages, information, and knowledge". However, "[t]he first author to distinguish among data, information, and knowledge and to also employ the term 'knowledge management' may have been American educator Nicholas L. Henry", in a 1974 journal article. === Data, information, knowledge, wisdom === Other early versions (prior to 1982) of the hierarchy that refer to a data tier include those of Chinese-American geographer Yi-Fu Tuan and sociologist-historian Daniel Bell.. In 1980, Irish-born engineer Mike Cooley invoked the same hierarchy in his critique of automation and computerization, in his book Architect or Bee?: The Human / Technology Relationship. Thereafter, in 1987, Czechoslovakia-born educator Milan Zeleny mapped the components of the hierarchy to knowledge forms: know-nothing, know-what, know-how, and know-why. Zeleny "has frequently been credited with proposing the [representation of DIKW as a pyramid ]... although he actually made no reference to any such graphical model." The hierarchy appears again in a 1988 address to the International Society for General Systems Research, by American organizational theorist Russell Ackoff, published in 1989. Subsequent authors and textbooks cite Ackoff's as the "original articulation" of the hierarchy or otherwise credit Ackoff with its proposal. Ackoff's version of the model includes an understanding tier (as Adler had, before him), interposed between knowledge and wisdom. Although Ackoff did not present the hierarchy graphically, he has also been credited with its representation as a pyramid. In 1989, Bell Labs veteran Robert W. Lucky wrote about the four-tier "information hierarchy" in the form of a pyramid in his book Silicon Dreams. In the same year as Ackoff presented his a

    Read more →
  • E-Science librarianship

    E-Science librarianship

    E-Science librarianship refers to a role for librarians in e-Science. == Early scholars == Early references to e-Science and librarianship involve information studies scholars researching cyberinfrastructure and emerging networked information and knowledge communities. Notably Christine Borgman, Professor and Presidential Chair in Information Studies at the University of California, Los Angeles (UCLA) was a key player in bringing e-Science, and the idea of networked knowledge communities, to the attention of the library profession. In 2004, as a visiting fellow at the Oxford Internet Institute, she conducted research and lectured publicly on e-Science, Digital Libraries, and Knowledge Communities. In 2007 Anna K. Gold, formerly of MIT and Cal Poly, San Luis Obispo, authored a series of articles in D-Lib Magazine that opened the door for academic libraries to begin exploring roles, skills, and strategies for engaging in e-Science: Cyberinfrastructure, Data, and Libraries, Part 1: A Cyberinfrastructure Primer for Librarians and Cyberinfrastructure, Data, and Libraries, Part 2: Libraries and the Data Challenge: Roles and Actions for Libraries. == Academic research and health sciences libraries == In 2007, the Association of Research Libraries (ARL) e-Science task force issued its report on e-Science and librarianship. The ARL's report encouraged its member libraries to position themselves to engage with researchers involved in e-Science (eScience) by cultivating new research support strategies and developing their digital scholarship infrastructure. E-Science has multiple attributes; Tony and Jessie Hey framed e-Science for the library community by characterizing it as a research methodology: "e-Science is not a new scientific discipline in its own right: e-Science is shorthand for the set of tools and technologies required to support collaborative, networked science". In addition to academic libraries' interests in providing support for their researchers engaging in e-Science, the health sciences library community also emerged as a major proponent for creating librarian positions for supporting the information needs of large-scale, networked, research collaborations on their campuses. Neil Rambo, current director of NYU's Health Sciences Library and former director of University of Washington Health Sciences Library, was the first to use the term in the Journal of the Medical Library Association, in his 2009 editorial e-Science and the Biomedical Library. Rambo's definition of e-Science highlighted the potential e-Science held for creating data as a research product: "E-science is a new research methodology, fueled by networked capabilities and the practical possibility of gathering and storing vast amounts of data." In response to this article the University of Massachusetts Medical School Lamar Soutter Library and National Network of Libraries of Medicine, New England Region encouraged health sciences libraries to cooperate to identify skills and develop a program for training e-Science Librarians. Then, in 2013, Shannon Bohle, an archivist who was employed in the library at Cold Spring Harbor Laboratory, an NCI-designated basic cancer research facility, used experience gained there and previous papers and presentations about preserving scientific archival materials to expand the traditional definition of e-Science by including the terms, principles, and practices used in archival science. These included in the definition the "long-term storage and accessibility of all materials generated through the scientific process," as well as examples of material types traditionally preserved in archives, like "electronic/digitized laboratory notebooks, raw and fitted data sets, manuscript production and draft versions, pre-prints," as well as library materials ("print and/or electronic publications"). == Roles == Many areas of science are about to be transformed by the availability of vast amounts of new scientific data that can potentially provide insights at a level of detail never before envisaged. However, this new data dominant era brings new challenges for the scientists and they will need the skills and technologies both of computer scientists and of the library community to manage, search and curate these new data resources. Libraries will not be immune from change in this new world of research. Karen Williams identifies roles in the following areas for librarians in the developing world of e-Science. Campus Engagement Content/Collection Development and Management Teaching and Learning Scholarly Communication E-Scholarship and Digital Tools Reference/Help Services Outreach Fund Raising Exhibit and Event Planning Leadership == Challenges for research libraries == E-science tends toward inter- and multidisciplinary approaches that depend on computation and computer science. Research libraries have traditionally been discipline focused and, although increasingly technologically sophisticated, do not have systems of the scale or complexity of the e-science environment. E-science is data intensive, but research libraries have not typically been responsible for scientific data. E-science is frequently conducted in a team context, often distributed across multiple institutions and on a global scale. The primary constituency of libraries generally comprises those affiliated with the local institution. Licenses for electronic content are typically restricted to a particular institutional community, and the infrastructure to move institutional licenses into a multi-institutional environment is not well developed. E-science challenges all these traditional paradigms of research library organization and services. == Skills == Garritano & Carlson were among the first to outline a skill set for librarians seeking to support the data needs of e-Science; they identified five skill categories librarians new to this area should expect to adapt or develop when participating on such projects: Library and information science expertise Subject expertise Partnerships and outreach (both internal and external) Participating in sponsored research Balancing workload An example of librarians reconfiguring traditional librarian skills to meet the needs of researchers engaging in e-Science is Witt & Carlson's adaptation of the traditional reference interview into a "data interview" in order to provide effective data management and e-Science services. This interview consists of ten practical queries necessary for understanding the provenance and expectations for the preservation of datasets typical of e-Science that also help illustrate some of the educational tools and skills needed by a librarian new to e-Science. "What is the story of the data? What form and format are the data in? What is the expected lifespan of the dataset? How could the data be used, reused, and repurposed? How large is the dataset, and what is its rate of growth? Who are the potential audiences for the data? Who owns the data? Does the dataset include any sensitive information? What publications or discoveries have resulted from the data? How should the data be made accessible?" == Resources == In 2009 the Lamar Soutter Library at the University of Massachusetts Medical School (UMMS) and the National Network of Libraries of Medicine, New England Region (NN/LM NER) funded an e-Science program for building the skills highlighted above for librarians. Elaine Russo Martin, Director of Library Services at the Lamar Soutter Library and Director of the NN/LM NER developed this comprehensive e-Science program to build librarians' subject expertise in the sciences, developing their data management skills, and their familiarity with cyberinfrastructure and e-Science. Three major products of this program are the e-Science web portal for librarians, the E-Science Symposium, and the New England Collaborative Data Management Curriculum (NECDMC). This portal includes educational resources for specific tools and subject/discipline tutorials and modules to assist librarians new to e-Science. UMMS and NN/LM NER also publish an open access journal called the Journal of eScience Librarianship.

    Read more →
  • AlphaTensor

    AlphaTensor

    AlphaTensor is an artificial intelligence system developed by DeepMind for discovering efficient matrix multiplication algorithms using reinforcement learning. Introduced in 2022, the system was based on AlphaZero and formulated the search for matrix multiplication algorithms as a single-player game called TensorGame. AlphaTensor was designed to search for new ways to multiply matrices with fewer scalar multiplication operations. Matrix multiplication is a fundamental operation in linear algebra, numerical analysis, scientific computing, computer graphics, and machine learning. The system discovered thousands of matrix multiplication algorithms, including algorithms that rediscovered known human-designed methods and others that improved on previously known results for particular matrix sizes and mathematical settings. == Background == Matrix multiplication is one of the basic operations in numerical computing. The standard algorithm for multiplying two square matrices has cubic time complexity, while faster algorithms such as the Strassen algorithm reduce the number of multiplication operations by using more complex algebraic decompositions. Finding optimal matrix multiplication algorithms can be difficult because it involves searching through a large space of possible tensor decompositions. AlphaTensor approached this problem by representing algorithm discovery as TensorGame, in which each move corresponds to an operation that reduces a tensor representing matrix multiplication. The goal of the game is to find a low-rank decomposition of the matrix multiplication tensor, corresponding to an efficient multiplication algorithm. == Development == AlphaTensor was developed by DeepMind and described in a paper published in Nature in October 2022. The system built on the reinforcement-learning approach used in AlphaZero, which had previously been applied to games such as Go, chess, and shogi. Unlike those games, TensorGame involved a very large search space, requiring changes to the AlphaZero-style search method and neural network architecture. DeepMind released source code and discovered algorithms associated with the publication through a public GitHub repository. == Results == AlphaTensor discovered matrix multiplication algorithms over both standard arithmetic and finite fields. One widely reported result was a method for multiplying 4 × 4 matrices over the field with two elements using 47 multiplication operations, improving on the 49 operations required by applying Strassen's algorithm recursively in that setting. The system also found algorithms optimized for particular computer hardware, including algorithms designed for graphics processing units and Tensor Processing Units. DeepMind stated that some of the hardware-specific algorithms improved practical execution time compared with commonly used algorithms on the tested hardware. == Significance == AlphaTensor was described as an example of using machine learning not only to apply existing algorithms, but to assist in discovering new ones. The work was connected to broader research in algorithm discovery, automated machine learning, program synthesis, and computational complexity theory, especially the open problem of determining the optimal complexity of matrix multiplication. AlphaTensor later became part of a broader group of Google DeepMind systems for algorithm and mathematical discovery, alongside systems such as AlphaDev and AlphaEvolve.

    Read more →
  • Sparrow (chatbot)

    Sparrow (chatbot)

    Sparrow is a chatbot developed by the artificial intelligence research lab DeepMind, a subsidiary of Alphabet Inc. It is designed to answer users' questions correctly, while reducing the risk of unsafe and inappropriate answers. One motivation behind Sparrow is to address the problem of language models producing incorrect, biased or potentially harmful outputs. Sparrow is trained using human judgements, in order to be more “Helpful, Correct and Harmless” compared to baseline pre-trained language models. The development of Sparrow involved asking paid study participants to interact with Sparrow, and collecting their preferences to train a model of how useful an answer is. To improve accuracy and help avoid the problem of hallucinating incorrect answers, Sparrow has the ability to search the Internet using Google Search in order to find and cite evidence for any factual claims it makes. To make the model safer, its behaviour is constrained by a set of rules, for example "don't make threatening statements" and "don't make hateful or insulting comments", as well as rules about possibly harmful advice, and not claiming to be a person. During development study participants were asked to converse with the system and try to trick it into breaking these rules. A 'rule model' was trained on judgements from these participants, which was used for further training. Sparrow was introduced in a paper in September 2022, titled "Improving alignment of dialogue agents via targeted human judgements"; however, the bot was not released publicly. DeepMind CEO Demis Hassabis said DeepMind is considering releasing Sparrow for a "private beta" some time in 2023. == Training == Sparrow is a deep neural network based on the transformer machine learning model architecture. It is fine-tuned from DeepMind's Chinchilla AI pre-trained large language model (LLM), which has 70 Billion parameters. Sparrow is trained using reinforcement learning from human feedback (RLHF), although some supervised fine-tuning techniques are also used. The RLHF training utilizes two reward models to capture human judgements: a “preference model” that predicts what a human study participant would prefer and a “rule model” that predicts if the model has broken one of the rules. == Limitations == Sparrow's training data corpus is mainly in English, meaning it performs worse in other languages. When adversarially probed by study participants it breaks the rules 8% of the time; however, this is still three times lower than the baseline prompted pre-trained model (Chinchilla).

    Read more →
  • Metadirectory

    Metadirectory

    A metadirectory system provides for the flow of data between one or more directory services and databases in order to maintain synchronization of that data. It is an important part of identity management systems. The data being synchronized typically are collections of entries that contain user profiles and possibly authentication or policy information. Most metadirectory deployments synchronize data into at least one LDAP-based directory server, to ensure that LDAP-based applications such as single sign-on and portal servers have access to recent data, even if the data is mastered in a non-LDAP data source. Metadirectory products support filtering and transformation of data in transit. Most identity management suites from commercial vendors include a metadirectory product, or a user provisioning product.

    Read more →
  • Virtual data room

    Virtual data room

    A virtual data room (sometimes called a VDR or Deal Room) is an online repository of information that is used for the storing and distribution of documents. In many cases, a virtual data room is used to facilitate the due diligence process during an M&A transaction, loan syndication, or private equity and venture capital transactions. This due diligence process has traditionally used a physical data room to accomplish the disclosure of documents. For reasons of cost, efficiency and security, virtual data rooms have widely replaced the more traditional physical data room. A virtual data room is an extranet to which the bidders and their advisers are given access via the internet. An extranet is essentially a website with limited controlled access, using a secure log-on supplied by the vendor, which can be disabled at any time, by the vendor, if a bidder withdraws. Much of the information released is confidential and restrictions are applied to the viewer's ability to release this to third parties (by means of forwarding, copying or printing). This can be effectively applied to protect the data using digital rights management. The virtual data room provides access to secure documents for authorized users through a dedicated web site, or through secure agent applications. In the process of mergers and acquisitions the data room is set up as part of the central repository of data relating to companies or divisions being acquired or sold. The data room enables the interested parties to view information relating to the business in a controlled environment where confidentiality can be preserved. Conventionally this was achieved by establishing a supervised, physical data room in secure premises with controlled access. In most cases, with a physical data room, only one bidder team can access the room at a time. A virtual data room is designed to have the same advantages as a conventional data room (controlling access, viewing, copying and printing, etc.) with fewer disadvantages. Due to their increased efficiency, many businesses and industries have moved to using virtual data rooms instead of physical data rooms. In 2006, a spokesperson for a company which sets up virtual deal rooms was reported claiming that the process reduced the bidding process by about thirty days compared to physical data rooms. In the process of startup fundraising, a virtual data room is set up to be a central location for key data, documents, and financials. These are shared with venture capital and angel investors and allows them to streamline due diligence. == Application == Any business dealing with private data can apply VDRs when secure transaction processing is required. This includes financial institutions that need to negotiate confidential customer information without involving third parties. VDRs have traditionally been used for IPOs and real estate asset management. Technology companies may use them to exchange and review code or confidential data needed for operations. The same is true for clients, who entrust their valuable code only to the most qualified people in the organisation. The code is not something that can be printed out and brought in a folder. It resides on a computer and must be used together. VDR can find application in any business that manages data in the form of documents, especially law firms, financial advisers or the B2B sector. The latter work with documents that must always be handled and controlled confidentially, and it is difficult to store them securely when they are on a server that other people can access. In addition, in B2B, it is important to close the deal as quickly as possible: the average sales cycle is one to three months. VDR can be compared to a locked filing cabinet where all those folders and documents are kept. It automates the mathematics of pricing to prevent revenue leakage, and initially integrates CRM to ensure accurate synchronisation of all account data, which is important for B2B in particular and sales in general. While virtual data rooms offer many advantages, they are not suitable for every industry. For example, some governments may decide to continue using physical data rooms for highly confidential information sharing. The damage from potential cyberattacks and data breaches exceeds the benefits offered by virtual data rooms. In such cases, the use of VDRs is not considered. Data breaches have particularly affected the US healthcare system from March 2021 to March 2022 - according to IBM Security the cost of the breach was a record high of $10.1 million.

    Read more →
  • Knuth–Plass line-breaking algorithm

    Knuth–Plass line-breaking algorithm

    The Knuth–Plass algorithm is a line-breaking algorithm designed for use in Donald Knuth's typesetting program TeX. It integrates the problems of text justification and hyphenation into a single algorithm by using a discrete dynamic programming method to minimize a loss function that attempts to quantify the aesthetic qualities desired in the finished output. The algorithm works by dividing the text into a stream of three kinds of objects: boxes, which are non-resizable chunks of content, glue, which are flexible, resizeable elements, and penalties, which represent places where breaking is undesirable (or, if negative, desirable). The loss function, known as "badness", is defined in terms of the deformation of the glue elements, and any extra penalties incurred through line breaking. Making hyphenation decisions follows naturally from the algorithm, but the choice of possible hyphenation points within words, and optionally their preference weighting, must be performed first, and that information inserted into the text stream in advance. Knuth and Plass' original algorithm does not include page breaking, but may be modified to interface with a pagination algorithm, such as the algorithm designed by Plass in his PhD thesis. Typically, the cost function for this technique should be modified so that it does not count the space left on the final line of a paragraph; this modification allows a paragraph to end in the middle of a line without penalty. The same technique can also be extended to take into account other factors such as the number of lines or costs for hyphenating long words. == Computational complexity == A naive brute-force exhaustive search for the minimum badness by trying every possible combination of breakpoints would take an impractical O ( 2 n ) {\displaystyle O(2^{n})} time. The classic Knuth-Plass dynamic programming approach to solving the minimization problem is a worst-case O ( n 2 ) {\displaystyle O(n^{2})} algorithm but usually runs much faster, in close to linear time. Solving for the Knuth-Plass optimum can be shown to be a special case of the convex least-weight subsequence problem, which can be solved in O ( n ) {\displaystyle O(n)} time. Methods to do this include the SMAWK algorithm. == Simple example of minimum raggedness metric == For the input text AAA BB CC DDDDD with line width 6, a greedy algorithm that puts as many words on a line as possible while preserving order before moving to the next line, would produce: ------ Line width: 6 AAA BB Remaining space: 0 CC Remaining space: 4 DDDDD Remaining space: 1 The sum of squared space left over by this method is 0 2 + 4 2 + 1 2 = 17 {\displaystyle 0^{2}+4^{2}+1^{2}=17} . However, the optimal solution achieves the smaller sum 3 2 + 1 2 + 1 2 = 11 {\displaystyle 3^{2}+1^{2}+1^{2}=11} : ------ Line width: 6 AAA Remaining space: 3 BB CC Remaining space: 1 DDDDD Remaining space: 1 The difference here is that the first line is broken before BB instead of after it, yielding a better right margin and a lower cost 11.

    Read more →
  • HTK Limited

    HTK Limited

    HTK Limited is a software-as-a-service company that provides mobile phone messaging and IVR services. Founded in 1996, HTK is headquartered in Ipswich, Suffolk, UK. HTK provide mass notification services. Specifically, the "Police Direct" messaging service to Suffolk and Norfolk Constabularies. In 2010 the HTK Horizon SaaS platform was selected by the Scottish Environment Protection Agency (SEPA) for their Floodline Warnings Direct service. == History == HTK was founded in 1996 by Marlon Bowser and Adrian Gregory and from the outset focused on what has now become commonly known as Software-as-a-Service. in 2004, according to the Deloitte Fast 50 (UK), HTK was the 17th fastest growing company in the East of England. In 2005 The Times listed HTK 65th nationally and 4th in the East of England in the Sunday Times & Microsoft "Tech Track 100" awards. In 2009 the company was approved as a supplier to UK Government under a new framework agreement. In 2010 HTK launched version 2.2 of its Horizon platform, with a feature set that signals a shift from mass notification into the customer service automation market.

    Read more →
  • Personal, Inc.

    Personal, Inc.

    Personal (also referred to as Personal.com or Personal, Inc.) was a consumer personal data service and identity management system for individuals to aggregate, manage and reuse their own data. It merged with digi.me in August 2017, a business in Europe that has the same business model. The combined company is called digi.me. One of its product lines, a collaborative data management and information security solution for the workplace called TeamData, was spun off as a new company as a result of the merger. == History == Personal was founded in 2009 in Washington, DC by the management team that built The Map Network, a location data and mapping platform that was acquired by Nokia/NAVTEQ in 2006. Personal was the first online consumer-facing company to be named an Ambassador for Privacy by Design for its technical, business and legal commitments to providing users with control over the data they store in Personal's service. Called a “life management platform” by The Economist and a “personal encrypted cloud service” by TIME for its user-centric approach to data, the company has been associated with both the Infomediary model originated in 1999 by John Hagel III and Mark Singer, as well as the vendor relationship management (VRM) model developed by Doc Searls. Personal raised $30m in funding to develop its platform and products from such leading investors as Steve Case's Revolution Ventures, Grotech Ventures, Allen & Company, Ted Leonsis, Neil Ashe, Jonathan Miller, Bill Miller of Legg Mason, Esther Dyson of EDventures, and Eric C. Anderson. The company received recognition for its user agreement, called the Owner Data Agreement, which acted like a reverse license agreement when data was shared between registered parties and emphasized that data ownership resides with the user. Doc Searls wrote in The Intention Economy: When Customers Take Charge that the Owner Data Agreement “had no precedent and modeled a new legal position, both for vendors and for intermediaries.” Personal was early to embrace “small data,” which it defines as “big data for the benefit of individuals.” The term “small data” may have been originally coined by Jeremie Miller of Sing.ly, who mentioned it in a talk at the Web 2.0 Summit in November 2011 and is cited in The Intention Economy. In 2011, Personal was a part of the first group of companies to join the Personal Data Ecosystem Consortium's Startup Circle. A Small Data Meetup group has also formed in New York City, bringing together technology, legal and business experts to exchange ideas about user-centric and user-driven models for internet products and services. Personal has been included in case studies by Ctrl-Shift and Forrester regarding Personal Data Stores and Personal Identity Management. In 2011, Personal received the Innovator Spotlight Award at Privacy Identity Innovation Conference (pii2011) and participated in the Technology Showcase at pii2012. In 2012, TechHive named Personal as one of the top five apps or web services of SXSW. Personal won the 2013 Campus Technology Innovators Award with Lone Star College in July 2013. Personal was included in a list of Executive Travel Magazine's favorite travel apps for 2013 in its May/June issue. In 2013, Personal was also included as part of NYU GovLab's Open Data 500 and was named by J. Walter Thompson as one of 100 things to watch for in 2014. In 2015, the National Law Journal named Company Chief Policy Officer and General Counsel, Joshua P. Galper, as one of their 50 "Cybersecurity & Privacy Trailblazers." == Products and services == === Overview === The Personal Platform was a privacy- and security-by-design platform for individuals to manage and reuse their own data and information. The Fill It app was a 1-click form-filling solution for web and mobile logins, checkouts and forms, and the Data Vault app served as the main cloud-based repository for a user's data. Personal helped individuals take control and benefit from their information while knowing that the information in their Data Vault remained legally theirs and could not be used without their permission. === Data Vault with Cloud Sync === Personal spent two years building the Personal Platform before launching its Data Vault product in beta in November 2011. Following Privacy by Design principles, Personal only enabled users to see or share the sensitive data and all the files they stored in their Data Vault. Such information was encrypted, and could only be decrypted with a user's password. Only users could choose and know their passwords to their vault because Personal did not store user passwords – and therefore could not reset them without deleting a user's sensitive data and all files stored in their vault. All Personal apps and services were linked to a user's private Data Vault. The Data Vault featured automatic synchronization of data and files added on any device logged into Personal. It also featured a “Secure Share” function that created a live, private network, allowing registered users to share access to data and files through an exchange of encrypted keys without the risk of transmitting the data or files through non-secure, direct means. It also allowed users to immediately update data across their own network and revoke access to it when they choose. Fast Company called the Data Vault “a tool that will simplify our lives.” Personal launched its Android app on November 30, 2011. The iOS Data Vault app was released on May 7, 2012. Personal officially launched its application programming interface (APIs) on October 2, 2012 at the Mashery Business of APIs Conference. A review by CNET highlighted the challenges of getting people to trust such a new service with their sensitive data and spending the time required entering enough data to make it useful. === Fill It App and Form Index === When the Data Vault was launched in November 2011, Mashable posed the question: “Never Fill Out a Form Again?” The World Economic Forum in its February 2013 report highlighted the possibility of saving 10 billion hours globally “and improv[ing] the delivery of public and private sector services” through automated form-filling tools, specifically citing Personal's Fill It app. In January 2013, Personal launched Fill It in beta as a web bookmarklet for automatic form-filling. On June 11, 2014, Personal released Fill It as a web extension and announced that it was publishing an index of over 140,000 1-click online forms at www.fillit.com. The company also announced that a mobile version of the product will launch later in the year. According to a story in Tech Cocktail about the launch, Personal's “web extension and mobile app are able to support over 1,200 different types of reusable data, even enabling them to unlock more confidential information so they can complete longer forms, including patient registrations, job applications, event registrations, school admissions, insurance and bank applications, and government forms.” In November 2014, a mobile version of Fill It was launched that could autofill mobile forms using APIs. Personal's form portal ultimately indexed more than 500,000 forms with three components, which, together, allowed data to be captured and reused across any of the forms: (1) a form graph, which mapped individual form fields to the Personal ontology; (2) a semantic layer, which determined how data was required on a form (e.g. one field vs. three fields for a U.S. telephone number); and (3) a correlations graph, which helped individuals match their specific data to a form without looking at the data value (e.g. knowing which phone number is a mobile phone number, which address is a billing address, or that a person uses their middle name as a first name on most forms). === Monetizing personal data === With the initial public offering of Facebook in May 2012, there was media interest in the question of the monetary value of personal data and whether tools and services might emerge to help consumers monetize their own data. Personal was frequently cited as a company that could potentially offer such a service. Articles and pieces focusing on this subject have appeared in The New York Times, AdWeek, the MIT Technology Review, and on CNN and National Public Radio. Company Co-founder and CEO Shane Green was quoted as saying that “the average American consumer would soon be able to realize over $1,000 per year” by granting limited, anonymous access to their data to marketers, but that figure was never supported by Green or the company. === Launch of TeamData === In May 2016, Personal shifted its product focus to TeamData, which focuses on the problem of securing and collaboratively managing data in the workplace. It is now a separate business.

    Read more →
  • Operational database

    Operational database

    Operational database management systems (also referred to as OLTP databases or online transaction processing databases), are used to update data in real-time. These types of databases allow users to do more than simply view archived data. Operational databases allow you to modify that data (add, change or delete data), doing it in real-time. OLTP databases provide transactions as main abstraction to guarantee data consistency that guarantee the so-called ACID properties. Basically, the consistency of the data is guaranteed in the case of failures and/or concurrent access to the data. == History == Since the early 1990s, the operational database software market has been largely taken over by SQL engines. In 2014, the operational DBMS market (formerly OLTP) was evolving dramatically, with new, innovative entrants and incumbents supporting the growing use of unstructured data and NoSQL DBMS engines, as well as XML databases and NewSQL databases. NoSQL databases typically have focused on scalability and have renounced to data consistency by not providing transactions as OLTP system do. Operational databases are increasingly supporting distributed database architecture that can leverage distribution to provide high availability and fault tolerance through replication and scale out ability. The growing role of operational databases in the IT industry is moving fast from legacy databases to real-time operational databases capable to handle distributed web and mobile demand and to address Big data challenges. Recognizing this, Gartner started to publish the Magic Quadrant for Operational Database Management Systems in October 2013. == List of operational databases == Notable operational databases include: == Use in business == Operational databases are used to store, manage and track real-time business information. For example, a company might have an operational database used to track warehouse/stock quantities. As customers order products from an online web store, an operational database can be used to keep track of how many items have been sold and when the company will need to reorder stock. An operational database stores information about the activities of an organization, for example customer relationship management transactions or financial operations, in a computer database. Operational databases allow a business to enter, gather, and retrieve large quantities of specific information, such as company legal data, financial data, call data records, personal employee information, sales data, customer data, data on assets and many other information. An important feature of storing information in an operational database is the ability to share information across the company and over the Internet. Operational databases can be used to manage mission-critical business data, to monitor activities, to audit suspicious transactions, or to review the history of dealings with a particular customer. They can also be part of the actual process of making and fulfilling a purchase, for example in e-commerce. == Data warehouse terminology == In data warehousing, the term is even more specific: the operational database is the one which is accessed by an operational system (for example a customer-facing website or the application used by the customer service department) to carry out regular operations of an organization. Operational databases usually use an online transaction processing database which is optimized for faster transaction processing (create, read, update and delete operations). An operational database is the source for a data warehouse. Data from an operational database can be loaded into an operational data store at a data warehouse before the data is processed into the data warehouse.

    Read more →