AI App Gemini

AI App Gemini — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.

  • IPUMS

    IPUMS

    IPUMS, originally the Integrated Public Use Microdata Series, is the world's largest individual-level population database. IPUMS consists of microdata samples from United States (IPUMS-USA) and international (IPUMS-International) census records, as well as data from U.S. and international surveys. The records are converted into a consistent format and made available to researchers through a web-based data dissemination and analysis system. IPUMS is housed at the Institute for Social Research and Data Innovation (ISRDI), an interdisciplinary research center at the University of Minnesota, under the direction of Professor Steven Ruggles. == Description == IPUMS includes all persons enumerated in the United States censuses from 1850 to 1950 (though, the 1890 census is missing because it was destroyed in a fire) and from the American Community Survey since 2000 and the Current Population Survey since 1962. IPUMS includes household-level data for United States Censuses from 1790 to 1840, due to the first six censuses only including the name of the head of household, with tallied household totals following. IPUMS provides consistent variable names, coding schemes, and documentation across all the samples, facilitating the analysis of long-term change. IPUMS-International includes countries from Africa, Asia, Europe, and Latin America for 1960 forward. The database currently includes more than a billion individuals enumerated in 365 censuses from 94 countries around the world. IPUMS-International converts census microdata for multiple countries into a consistent format, allowing for comparisons across countries and time periods. Special efforts are made to simplify use of the data while losing no meaningful information. Comprehensive documentation is provided in a coherent form to facilitate comparative analyses of social and economic change. Additional databases in the IPUMS family include the: North Atlantic Population Project (NAPP) IPUMS National Historical Geographic Information System (NHGIS) IPUMS Health Surveys IPUMS Global Health IPUMS Time Use The Journal of American History described the effort as "One of the great archival projects of the past two decades." Liens Socio, the French portal for the social sciences, gave IPUMS the only “best site” designation that has gone to any non-French website, writing “IPUMS est un projet absolument extraordinaire...époustouflante [mind-blowing]!” The official motto of IPUMS is "use it for good, never for evil." All public IPUMS data and documentation are available online free of charge.

    Read more →
  • Seccomp

    Seccomp

    seccomp (short for secure computing) is a computer security facility in the Linux kernel. seccomp allows a process to make a one-way transition into a "secure" state where it cannot make any system calls except exit(), sigreturn(), read() and write() to already-open file descriptors. Should it attempt any other system calls, the kernel will either just log the event or terminate the process with SIGKILL or SIGSYS. In this sense, it does not virtualize the system's resources but isolates the process from them entirely. seccomp mode is enabled via the prctl(2) system call using the PR_SET_SECCOMP argument, or (since Linux kernel 3.17) via the seccomp(2) system call. seccomp mode used to be enabled by writing to a file, /proc/self/seccomp, but this method was removed in favor of prctl(). In some kernel versions, seccomp disables the RDTSC x86 instruction, which returns the number of elapsed processor cycles since power-on, used for high-precision timing. seccomp-bpf is an extension to seccomp that allows filtering of system calls using a configurable policy implemented using Berkeley Packet Filter rules. It is used by OpenSSH and vsftpd as well as the Google Chrome/Chromium web browsers on ChromeOS and Linux. (In this regard seccomp-bpf achieves similar functionality, but with more flexibility and higher performance, to the older systrace—which seems to be no longer supported for Linux.) Some consider seccomp comparable to OpenBSD pledge(2) and FreeBSD capsicum(4). == History == seccomp was first devised by Andrea Arcangeli in January 2005 for use in public grid computing and was originally intended as a means of safely running untrusted compute-bound programs. It was merged into the Linux kernel mainline in kernel version 2.6.12, which was released on March 8, 2005. == Software using seccomp or seccomp-bpf == Android uses a seccomp-bpf filter in the zygote since Android 8.0 Oreo. systemd's sandboxing options are based on seccomp. QEMU, the Quick Emulator, the core component to the modern virtualization together with KVM uses seccomp on the parameter --sandbox Docker – software that allows applications to run inside of isolated containers. Docker can associate a seccomp profile with the container using the --security-opt parameter. Arcangeli's CPUShare was the only known user of seccomp for a while. Writing in February 2009, Linus Torvalds expresses doubt whether seccomp is actually used by anyone. However, a Google engineer replied that Google is exploring using seccomp for sandboxing its Chrome web browser. Firejail is an open source Linux sandbox program that utilizes Linux namespaces, Seccomp, and other kernel-level security features to sandbox Linux and Wine applications. As of Chrome version 20, seccomp-bpf is used to sandbox Adobe Flash Player. As of Chrome version 23, seccomp-bpf is used to sandbox the renderers. Snap specify the shape of their application sandbox using "interfaces" which snapd translates to seccomp, AppArmor and other security constructs vsftpd uses seccomp-bpf sandboxing as of version 3.0.0. OpenSSH has supported seccomp-bpf since version 6.0. Mbox uses ptrace along with seccomp-bpf to create a secure sandbox with less overhead than ptrace alone. LXD, a Ubuntu "hypervisor" for containers Firefox and Firefox OS, which use seccomp-bpf Tor supports seccomp since 0.2.5.1-alpha Lepton, a JPEG compression tool developed by Dropbox uses seccomp Kafel is a configuration language, which converts readable policies into seccompb-bpf bytecode Subgraph OS uses seccomp-bpf Flatpak uses seccomp for process isolation Bubblewrap is a lightweight sandbox application developed from Flatpak minijail uses seccomp for process isolation SydBox uses seccomp-bpf to improve the runtime and security of the ptrace sandboxing used to sandbox package builds on Exherbo Linux distribution. File, a Unix program to determine filetypes, uses seccomp to restrict its runtime environment Zathura, a minimalistic document viewer, uses seccomp filter to implement different sandbox modes Tracker, a indexing and preview application for the GNOME desktop environment, uses seccomp to prevent automatic exploitation of parsing vulnerabilities in media files

    Read more →
  • Open-source software security

    Open-source software security

    Open-source software security is the measure of assurance or guarantee in the freedom from danger and risk inherent to an open-source software system. == Implementation debate == === Benefits === Proprietary software forces the user to accept the level of security that the software vendor is willing to deliver and to accept the rate that patches and updates are released. It is assumed that any compiler that is used creates code that can be trusted, but it has been demonstrated by Ken Thompson that a compiler can be subverted using a compiler backdoor to create faulty executables that are unwittingly produced by a well-intentioned developer. With access to the source code for the compiler, the developer has at least the ability to discover if there is any mal-intention. Kerckhoffs' principle is based on the idea that an enemy can steal a secure military system and not be able to compromise the information. His ideas were the basis for many modern security practices, and followed that security through obscurity is a bad practice. === Drawbacks === Simply making source code available does not guarantee review. An example of this occurring is when Marcus Ranum, an expert on security system design and implementation, released his first public firewall toolkit. At one time, there were over 2,000 sites using his toolkit, but only 10 people gave him any feedback or patches. Having a large amount of eyes reviewing code can "lull a user into a false sense of security". Having many users look at source code does not guarantee that security flaws will be found and fixed. == Metrics and models == There are a variety of models and metrics to measure the security of a system. These are a few methods that can be used to measure the security of software systems. === Number of days between vulnerabilities === It is argued that a system is most vulnerable after a potential vulnerability is discovered, but before a patch is created. By measuring the number of days between the vulnerability and when the vulnerability is fixed, a basis can be determined on the security of the system. There are a few caveats to such an approach: not every vulnerability is equally bad, and fixing a lot of bugs quickly might not be better than only finding a few and taking a little bit longer to fix them, taking into account the operating system, or the effectiveness of the fix. === Poisson process === The Poisson process can be used to measure the rates at which different people find security flaws between open and closed source software. The process can be broken down by the number of volunteers Nv and paid reviewers Np. The rates at which volunteers find a flaw is measured by λv and the rate that paid reviewers find a flaw is measured by λp. The expected time that a volunteer group is expected to find a flaw is 1/(Nv λv) and the expected time that a paid group is expected to find a flaw is 1/(Np λp). === Morningstar model === By comparing a large variety of open source and closed source projects a star system could be used to analyze the security of the project similar to how Morningstar, Inc. rates mutual funds. With a large enough data set, statistics could be used to measure the overall effectiveness of one group over the other. An example of such as system is as follows: 1 Star: Many security vulnerabilities. 2 Stars: Reliability issues. 3 Stars: Follows best security practices. 4 Stars: Documented secure development process. 5 Stars: Passed independent security review. === Coverity scan === Coverity in collaboration with Stanford University has established a new baseline for open-source quality and security. The development is being completed through a contract with the Department of Homeland Security. They are utilizing innovations in automated defect detection to identify critical types of bugs found in software. The level of quality and security is measured in rungs. Rungs do not have a definitive meaning, and can change as Coverity releases new tools. Rungs are based on the progress of fixing issues found by the Coverity Analysis results and the degree of collaboration with Coverity. They start with Rung 0 and currently go up to Rung 2. Rung 0 The project has been analyzed by Coverity's Scan infrastructure, but no representatives from the open-source software have come forward for the results. Rung 1 At rung 1, there is collaboration between Coverity and the development team. The software is analyzed with a subset of the scanning features to prevent the development team from being overwhelmed. Rung 2 There are 11 projects that have been analyzed and upgraded to the status of Rung 2 by reaching zero defects in the first year of the scan. These projects include: AMANDA, ntp, OpenPAM, OpenVPN, Overdose, Perl, PHP, Postfix, Python, Samba, and Tcl.

    Read more →
  • IT baseline protection

    IT baseline protection

    The IT baseline protection (German: IT-Grundschutz) approach from the German Federal Office for Information Security (BSI) is a methodology to identify and implement computer security measures in an organization. The aim is the achievement of an adequate and appropriate level of security for IT systems. To reach this goal the BSI recommends "well-proven technical, organizational, personnel, and infrastructural safeguards". Organizations and federal agencies show their systematic approach to secure their IT systems (e.g. Information Security Management System) by obtaining an ISO/IEC 27001 Certificate on the basis of IT-Grundschutz. == Overview baseline security == The term baseline security signifies standard security measures for typical IT systems. It is used in various contexts with somewhat different meanings. For example: Microsoft Baseline Security Analyzer: Software tool focused on Microsoft operating system and services security Cisco security baseline: Vendor recommendation focused on network and network device security controls Nortel baseline security: Set of requirements and best practices with a focus on network operators ISO/IEC 13335-3 defines a baseline approach to risk management. This standard has been replaced by ISO/IEC 27005, but the baseline approach was not taken over yet into the 2700x series. There are numerous internal baseline security policies for organizations, The German BSI has a comprehensive baseline security standard, that is compliant with the ISO/IEC 27000-series == BSI IT baseline protection == The foundation of an IT baseline protection concept is initially not a detailed risk analysis. It proceeds from overall hazards. Consequently, sophisticated classification according to damage extent and probability of occurrence is ignored. Three protection needs categories are established. With their help, the protection needs of the object under investigation can be determined. Based on these, appropriate personnel, technical, organizational and infrastructural security measures are selected from the IT Baseline Protection Catalogs. The Federal Office for Security in Information Technology's IT Baseline Protection Catalogs offer a "cookbook recipe" for a normal level of protection. Besides probability of occurrence and potential damage extents, implementation costs are also considered. By using the Baseline Protection Catalogs, costly security analyses requiring expert knowledge are dispensed with, since overall hazards are worked with in the beginning. It is possible for the relative layman to identify measures to be taken and to implement them in cooperation with professionals. The BSI grants a baseline protection certificate as confirmation for the successful implementation of baseline protection. In stages 1 and 2, this is based on self declaration. In stage 3, an independent, BSI-licensed auditor completes an audit. Certification process internationalization has been possible since 2006. ISO/IEC 27001 certification can occur simultaneously with IT baseline protection certification. (The ISO/IEC 27001 standard is the successor of BS 7799-2). This process is based on the new BSI security standards. This process carries a development price which has prevailed for some time. Corporations having themselves certified under the BS 7799-2 standard are obliged to carry out a risk assessment. To make it more comfortable, most deviate from the protection needs analysis pursuant to the IT Baseline Protection Catalogs. The advantage is not only conformity with the strict BSI, but also attainment of BS 7799-2 certification. Beyond this, the BSI offers a few help aids like the policy template and the GSTOOL. One data protection component is available, which was produced in cooperation with the German Federal Commissioner for Data Protection and Freedom of Information and the state data protection authorities and integrated into the IT Baseline Protection Catalog. This component is not considered, however, in the certification process. == Baseline protection process == The following steps are taken pursuant to the baseline protection process during structure analysis and protection needs analysis: The IT network is defined. IT structure analysis is carried out. Protection needs determination is carried out. A baseline security check is carried out. IT baseline protection measures are implemented. Creation occurs in the following steps: IT structure analysis (survey) Assessment of protection needs Selection of actions Running comparison of nominal and actual. === IT structure analysis === An IT network includes the totality of infrastructural, organizational, personnel, and technical components serving the fulfillment of a task in a particular information processing application area. An IT network can thereby encompass the entire IT character of an institution or individual division, which is partitioned by organizational structures as, for example, a departmental network, or as shared IT applications, for example, a personnel information system. It is necessary to analyze and document the information technological structure in question to generate an IT security concept and especially to apply the IT Baseline Protection Catalogs. Due to today's usually heavily networked IT systems, a network topology plan offers a starting point for the analysis. The following aspects must be taken into consideration: The available infrastructure, The organizational and personnel framework for the IT network, Networked and non-networked IT systems employed in the IT network. The communications connections between IT systems and externally, IT applications run within the IT network. === Protection needs determination === The purpose of the protection needs determination is to investigate what protection is sufficient and appropriate for the information and information technology in use. In this connection, the damage to each application and the processed information, which could result from a breach of confidentiality, integrity or availability, is considered. Important in this context is a realistic assessment of the possible follow-on damages. A division into the three protection needs categories "low to medium", "high" and "very high" has proved itself of value. "Public", "internal" and "secret" are often used for confidentiality. === Modelling === Heavily networked IT systems typically characterize information technology in government and business these days. As a rule, therefore, it is advantageous to consider the entire IT system and not just individual systems within the scope of an IT security analysis and concept. To be able to manage this task, it makes sense to logically partition the entire IT system into parts and to separately consider each part or even an IT network. Detailed documentation about its structure is prerequisite for the use of the IT Baseline Protection Catalogs on an IT network. This can be achieved, for example, via the IT structure analysis described above. The IT Baseline Protection Catalog’s' components must ultimately be mapped onto the components of the IT network in question in a modelling step. === Baseline security check === The baseline security check is an organisational instrument offering a quick overview of the prevailing IT security level. With the help of interviews, the status quo of an existing IT network (as modelled by IT baseline protection) relative to the number of security measures implemented from the IT Baseline Protection Catalogs are investigated. The result is a catalog in which the implementation status "dispensable", "yes", "partly", or "no" is entered for each relevant measure. By identifying not yet, or only partially, implemented measures, improvement options for the security of the information technology in question are highlighted. The baseline security check gives information about measures, which are still missing (nominal vs. actual comparison). From this follows what remains to be done to achieve baseline protection through security. Not all measures suggested by this baseline check need to be implemented. Peculiarities are to be taken into account! It could be that several more or less unimportant applications are running on a server, which have lesser protection needs. In their totality, however, these applications are to be provided with a higher level of protection. This is called the (cumulation effect). The applications running on a server determine its need for protection. Several IT applications can run on an IT system. When this occurs, the application with the greatest need for protection determines the IT system’s protection category. Conversely, it is conceivable that an IT application with great protection needs does not automatically transfer this to the IT system. This may happen because the IT system is configured redundantly, or because only an inconsequential part is running on it. This is called the (distribution effect). This is the case, fo

    Read more →
  • Software engineering professionalism

    Software engineering professionalism

    Software engineering professionalism is a movement to make software engineering a profession, with aspects such as degree and certification programs, professional associations, professional ethics, and government licensing. The field is a licensed discipline in Texas in the United States (Texas Board of Professional Engineers, since 2013), Engineers Australia(Course Accreditation since 2001, not Licensing), and many provinces in Canada. == History == In 1993 the IEEE and ACM began a joint effort called JCESEP, which evolved into SWECC in 1998 to explore making software engineering into a profession. The ACM pulled out of SWECC in May 1999, objecting to its support for the Texas professionalization efforts, of having state licenses for software engineers. ACM determined that the state of knowledge and practice in software engineering was too immature to warrant licensing, and that licensing would give false assurances of competence even if the body of knowledge were mature. The IEEE continued to support making software engineering a branch of traditional engineering. In Canada the Canadian Information Processing Society established the Information Systems Professional certification process. Also, by the late 1990s (1999 in British Columbia) the discipline of software engineering as a professional engineering discipline was officially created. This has caused some disputes between the provincial engineering associations and companies who call their developers software engineers, even though these developers have not been licensed by any engineering association. In 1999, the Panel of Software Engineering was formed as part of the settlement between Engineering Canada and the Memorial University of Newfoundland over the school's use of the term "software engineering" in the name of a computer science program. Concerns were raised over the inappropriate use of the name "software engineering" to describe non-engineering programs could lead to student and public confusion, and ultimately threaten public safety. The Panel issued recommendations to create a Software Engineering Accreditation Board, but the task force created to carry out the recommendations was unable to get the various stakeholders to agree to concrete proposals, resulting in separate accreditation boards. == Ethics == Software engineering ethics is a large field. In some ways it began as an unrealistic attempt to define bugs as unethical. More recently it has been defined as the application of both computer science and engineering philosophy, principles, and practices to the design and development of software systems. Due to this engineering focus and the increased use of software in mission critical and human critical systems, where failure can result in large losses of capital but more importantly lives such as the Therac-25 system, many ethical codes have been developed by a number of societies, associations and organizations. These entities, such as the ACM, IEEE, EGBC and Institute for Certification of Computing Professionals (ICCP) have formal codes of ethics. Adherence to the code of ethics is required as a condition of membership or certification. According to the ICCP, violation of the code can result in revocation of the certificate. Also, all engineering societies require conformance to their ethical codes; violation of the code results in the revocation of the license to practice engineering in the society's jurisdiction. These codes of ethics usually have much in common. They typically relate the need to act consistently with the client's interest, employer's interest, and most importantly the public's interest. They also outline the need to act with professionalism and to promote an ethical approach to the profession. A Software Engineering Code of Ethics has been approved by the ACM and the IEEE-CS as the standard for teaching and practicing software engineering. === Examples of codes of conduct === The following are examples of codes of conduct for Professional Engineers. These 2 have been chosen because both jurisdictions have a designation for Professional Software Engineers. Engineers and Geoscientists of British Columbia (EGBC): All members in the association's code of Ethics must ensure that the government, the public can rely on BC's professional engineers and Geoscientists to act at all times with fairness, courtesy and good faith to their employers, employee and customers, and to uphold the truth, honesty and trustworthiness, and to safe guard human life and the environment. This is just one of the many ways in which BC's Professional Engineers and Professional Geoscientists maintain their competitive edge in today's global marketplace. Association of Professional Engineers and Geoscientists of Alberta (APEGA): Different with British Columbia, the Alberta Government granted self governance to engineers, Geoscientists and geophysicists. All members in the APEGA have to accept legal and ethical responsibility for the work and to hold the interest of the public and society. The APEGA is a standards guideline of professional practice to uphold the protection of public interest for engineering, Geoscientists and geophysics in Alberta. === Opinions on ethics === Bill Joy argued that "better software" can only enable its privileged end users, make reality more power-pointy as opposed to more humane, and ultimately run away with itself so that "the future doesn't need us." He openly questioned the goals of software engineering in this respect, asking why it isn't trying to be more ethical rather than more efficient. In his book Code and Other Laws of Cyberspace, Lawrence Lessig argues that computer code can regulate conduct in much the same way as the legal code. Lessig and Joy urge people to think about the consequences of the software being developed, not only in a functional way, but also in how it affects the public and society as a whole. Overall, due to the youth of software engineering, many of the ethical codes and values have been borrowed from other fields, such as mechanical and civil engineering. However, there are many ethical questions that even these, much older, disciplines have not encountered. Questions about the ethical impact of internet applications, which have a global reach, have never been encountered until recently and other ethical questions are still to be encountered. This means the ethical codes for software engineering are a work in progress, that will change and update as more questions arise. == Independent licensing and certification exams == Since 2002, the IEEE Computer Society offered the Certified Software Development Professional (CSDP) certification exam (in 2015 this was replaced by several similar certifications). A group of experts from industry and academia developed the exam and maintained it. Donald Bagert, and at a later period Stephen Tockey headed the certification committee. Contents of the exam centered around the SWEBOK (Software Engineering Body of Knowledge) guide, with an additional emphasis on Professional Practices and Software Engineering Economics knowledge areas (KAs). The motivation was to produce a structure at an international level for software engineering's knowledge areas. == Criticism of licensing == Professional licensing has been criticized for many reasons. The field of software engineering is too immature Licensing would give false assurances of competence even if the body of knowledge were mature Software engineers would have to study years of calculus, physics, and chemistry to pass the exams, which is irrelevant to most software practitioners. Many (most?) computer science majors don't earn degrees in engineering schools, so they are probably unqualified to pass engineering exams. == Licensing by country == === United States === The Bureau of Labor Statistics (BLS) classifies computer software engineers as a subcategory of "computer specialists", along with occupations such as computer scientist, Programmer, Database administrator and Network administrator. The BLS classifies all other engineering disciplines, including computer hardware engineers, as engineers. Many states prohibit unlicensed persons from calling themselves an Engineer, or from indicating branches or specialties not covered licensing acts. In many states, the title Engineer is reserved for individuals with a Professional Engineering license indicating that they have shown minimum level of competency through accredited engineering education, qualified engineering experience, and engineering board's examinations. In April 2013 the National Council of Examiners for Engineering and Surveying (NCEES) began offering a Professional Engineer (PE) exam for Software Engineering. The exam was developed in association with the IEEE Computer Society. NCEES ended the exam in April 2019 due to lack of participation. The American National Society of Professional Engineers provides a model law and lobbies legislatures to adopt occ

    Read more →
  • Generative design

    Generative design

    Generative design is an iterative design process that uses software to generate outputs that fulfill a set of constraints iteratively adjusted by a designer. Whether a human, test program, or artificial intelligence, the designer algorithmically or manually refines the feasible region of the program's inputs and outputs with each iteration to fulfill evolving design requirements. By employing computing power to evaluate more design permutations than a human alone is capable of, the process is capable of producing an optimal design that mimics nature's evolutionary approach to design through genetic variation and selection. The output can be images, sounds, architectural models, animation, and much more. It is, therefore, a fast method of exploring design possibilities that is used in various design fields such as art, architecture, communication design, and product design. Generative design has become more important, largely due to new programming environments or scripting capabilities that have made it relatively easy, even for designers with little programming experience, to implement their ideas. Additionally, this process can create solutions to substantially complex problems that would otherwise be resource-exhaustive with an alternative approach, making it a more attractive option for problems with a large or unknown solution set. It is also facilitated with tools in commercially available CAD packages. Not only are implementation tools more accessible, but also tools leveraging generative design as a foundation. Recent advancements have led to the development of Deep Generative Design, a framework that integrates topology optimization with deep learning models, such as Generative Adversarial Networks (GANs). Unlike traditional evolutionary methods that primarily focus on engineering performance, this approach uses deep generative models to enhance aesthetic diversity and novelty while simultaneously satisfying engineering constraints. For instance, research by Oh et al. (2019) proposed a framework using Boundary Equilibrium GANs (BEGAN) to generate diverse design options which are then refined through density-based topology optimization, allowing for the exploration of complex design spaces that balance structural integrity with visual variation. In practice, generative design does not solely aim to produce a single optimal solution, but involves iteratively refining the design problem by modifying parameters, constraints, and evaluation criteria within a computational model, resulting in multiple design alternatives from which the designer selects. == Use in architecture == Generative design in architecture is an iterative design process that enables architects to explore a wider solution space with more possibility and creativity. Architectural design has long been regarded as a wicked problem. Compared with traditional top-down design approach, generative design can address design problems efficiently, by using a bottom-up paradigm that uses parametric-defined rules to generate complex solutions. The solution itself then evolves to a good, if not optimal, solution. The advantage of using generative design as a design tool is that it does not construct fixed geometries, but take a set of design rules that can generate an infinite set of possible design solutions. The generated design solutions can be more sensitive, responsive, and adaptive to the problem. Generative design involves rule definition and result analysis that are integrated with the design process. By defining parameters and rules, the generative approach is able to provide optimized solution for both structural stability and aesthetics. Possible design algorithms include cellular automata, shape grammar, genetic algorithm, space syntax, and most recently, artificial neural network. Due to the high complexity of the solution generated, rule-based computational tools, such as finite element method and topology optimisation, are preferred to evaluate and optimise the generated solution. The iterative process provided by computer software enables the trial-and-error approach in design, and involves architects interfering with the optimisation process. Historically precedent work includes Antoni Gaudí's Sagrada Família, which used rule based geometrical forms for structures, and Buckminster Fuller's Montreal Biosphere where the rules were designed to generate individual components, rather than the final product. More recent generative-design cases include Foster and Partners' Queen Elizabeth II Great Court, where the tessellated glass roof was designed using a geometric schema to define hierarchical relationships, and then the generated solution was optimized based on geometrical and structural requirements. == Use in sustainable design == Generative design in sustainable design is an effective approach addressing energy efficiency and climate change at the early design stage, recognizing buildings contribute to approximately one-third of global greenhouse gas emissions and 30%-40% of total building energy use. It integrates environmental principles with algorithms, enabling exploration of countless design alternatives to enhance energy performance, reduce carbon footprints, and minimize waste. A key feature of generative design in sustainable design is its ability to incorporate Building Performance Simulations (BPS) into the design process. Simulation programs such as EnergyPlus, Ladybug Tools,, and so on, combined with generative algorithms, can optimize design solutions for cost-effective energy use and zero-carbon building designs. For example, the GENE_ARCH system used a Pareto algorithm with building energy simulation for the whole building design optimization. Generative design has improved sustainable facade design, as illustrated by the algorithm of cellular automata and daylight simulations in adaptive facade design. In addition, genetic algorithms were used with radiation simulations for energy-efficient photo-voltaic (PV) modules on high-rise building facades. Generative design is also applied to life cycle analysis (LCA), as demonstrated by a framework using grid search algorithms to optimize exterior wall design for minimum environmental impact. Multi-objective optimization embraces multiple diverse sustainability goals, such as interactive kinetic louvers using biomimicry and daylight simulations to enhance daylight, visual comfort, and energy efficiency. The study of PV and shading systems can maximize on-site electricity, improve visual quality, and daylight performance. Artificial intelligence (AI) and machine learning (ML) further improve computation efficiency in complex climate-responsive sustainable design. One study employed reinforcement learning to identify the relationship between design parameters and energy use for a sustainable campus, while other studies tried hybrid algorithms, such as using the genetic algorithm and GANs to balance daylight illumination and thermal comfort under different roof conditions. Other popular AI tools were also integrated, including deep reinforcement learning (DRL) and computer vision (CV), to generate an urban block according to direct sunlight hours and solar heat gains. These AI-driven generative design methods enable faster simulations and design decision making, resulting in designs that are environmentally responsible. == Use in additive manufacturing == Additive manufacturing (AM) is a process that creates physical models directly from three-dimensional (3D) data by joining materials layer by layer. It is used in industries to produce a variety of end-use parts, which are final components designed for direct application in products or systems. AM provides design flexibility and enables material reduction in lightweight applications, such as aerospace, automotive, medical, and portable electronic devices, where minimizing weight is critical for performance. Generative design, one of the four key methods for lightweight design in AM, is commonly applied to optimize structures for specific performance requirements. Generative design can help create optimized solutions that balance multiple objectives, such as enhancing performance while minimizing cost. In design for additive manufacturing (DfAM), multi-objective topology optimization is used to generate a set of candidate solutions. Designers then assess these options using their expertise and key performance indicators (KPIs) to select the best option for implementation. However, integrating AM constraints (e.g., speed of build, materials, build envelope, and accuracy) into generative design remains challenging, as ensuring all solutions are valid is complex. Balancing multiple design objectives while limiting computational costs adds further challenges for designers. To overcome these difficulties, researchers proposed a generative design method with manufacturing validation to improve decision-making efficiency. This method starts with a cons

    Read more →
  • Distinguishable interfaces

    Distinguishable interfaces

    Distinguishable interfaces use computer graphic principles to automatically generate easily distinguishable appearance for computer data. Although the desktop metaphor revolutionized user interfaces, there is evidence that a spatial layout alone does little to help in locating files and other data; distinguishable appearance is also required. Studies have shown that average users have considerable difficulty finding files on their personal computers, even ones that they created the same day. Search engines do not always help, since it has been found that users often know of the existence of a file without being able to specify relevant search terms. On the contrary, people appear to incrementally search for files using some form of context. Recently researchers and web developers have argued that the problem is the lack of distinguishable appearance: in the traditional computer interface most objects and locations appear identical. This problem rarely occurs in the real world, where both objects and locations generally have easily distinguishable appearance. Discriminability was one of the recommendations in the ISO 9241-12 recommendation on presentation of information on visual displays (part of the overall report on Ergonomics of Human System Interaction), however it was assumed in that report that this would be achieved by manual design of graphical symbols. == VisualIDs, semanticons, and identicons == The mass availability of computer graphics supported the introduction of approaches that make better use of the brain's "visual hardware", by providing individual files and other abstract data with distinguishable appearance. This idea initially appeared in strictly academic VisualIDs and Semanticons works, but the web community has explored and rapidly adopted similar ideas, such as the Identicon. The VisualIDs project automatically generated icons for files or other data based on a hash of the data identifier, so the icons had no relation to the content or meaning of the data. It was argued not only that generating meaningful icons is unnecessary (their user study showed rapid learning of the arbitrary icons), but also that basing icons on content is actually incorrect ("contrasting visualization with visual identifiers"). The Semanticons project developed by Setlur et al. demonstrated an algorithm to create icons that reflect the content of files. In this work the name, location and content of a file are parsed and used to retrieve related image(s) from an image database. These are then processed using a Non-photorealistic rendering technique in order to generate graphical icons. Developer Don Park introduced the identicon library for making a visual icon from a hash of a data identifier. This initial public implementation has spawned a large number of implementations for various environments. In particular, identicons are now being used as default visual user identifiers (avatars) for several widely used systems. They are also used as a complement to Gravatars, which are pre-existing avatar images created or chosen by users, instead of automatically generated images. (see #External links). == Current research == While current web practice has followed the semantics-free approach of VisualIDs, recent research has followed the semantics-based approach of Semanticons. Examples include using data mining principles to automatically create "intelligent icons" that reflect the contents of files and creating icons for music files that reflect audio characteristics or affective content.

    Read more →
  • Hexagonal sampling

    Hexagonal sampling

    A multidimensional signal is a function of M independent variables where M ≥ 2 {\displaystyle M\geq 2} . Real world signals, which are generally continuous time signals, have to be discretized (sampled) in order to ensure that digital systems can be used to process the signals. It is during this process of discretization where sampling comes into picture. Although there are many ways of obtaining a discrete representation of a continuous time signal, periodic sampling is by far the simplest scheme. Theoretically, sampling can be performed with respect to any set of points. But practically, sampling is carried out with respect to a set of points that have a certain algebraic structure. Such structures are called lattices. Mathematically, the process of sampling an N {\displaystyle N} -dimensional signal can be written as: w ( t ^ ) = w ( V . n ^ ) {\displaystyle w({\hat {t}})=w(V.{\hat {n}})} where t ^ {\displaystyle {\hat {t}}} is continuous domain M-dimensional vector (M-D) that is being sampled, n ^ {\displaystyle {\hat {n}}} is an M-dimensional integer vector corresponding to indices of a sample, and V is an N × N {\displaystyle N\times N} sampling matrix. == Motivation == Multidimensional sampling provides the opportunity to look at digital methods to process signals. Some of the advantages of processing signals in the digital domain include flexibility via programmable DSP operations, signal storage without the loss of fidelity, opportunity for encryption in communication, lower sensitivity to hardware tolerances. Thus, digital methods are simultaneously both powerful and flexible. In many applications, they act as less expensive alternatives to their analog counterparts. Sometimes, the algorithms implemented using digital hardware are so complex that they have no analog counterparts. Multidimensional digital signal processing deals with processing signals represented as multidimensional arrays such as 2-D sequences or sampled images.[1] Processing these signals in the digital domain permits the use of digital hardware where in signal processing operations are specified by algorithms. As real world signals are continuous time signals, multidimensional sampling plays a crucial role in discretizing the real world signals. The discrete time signals are in turn processed using digital hardware to extract information from the signal. == Preliminaries == === Region of Support === The region outside of which the samples of the signal take zero values is known as the Region of support (ROS). From the definition, it is clear that the region of support of a signal is not unique. === Fourier transform === The Fourier transform is a tool that allows us to simplify mathematical operations performed on the signal. The transform basically represents any signal as a weighted combination of sinusoids. The Fourier and the inverse Fourier transform of an M-dimensional signal can be defined as follows: X a ( Ω ^ ) = ∫ − ∞ + ∞ x a ( t ^ ) e − j Ω ^ T t ^ d t ^ {\displaystyle X_{a}({\hat {\Omega }})=\int _{-\infty }^{+\infty }\!x_{a}({\hat {t}})e^{-j{\hat {\Omega }}^{T}{\hat {t}}}d{\hat {t}}} x a ( t ^ ) = 1 2 π M ∫ − ∞ + ∞ X ( Ω ^ ) e ( j Ω ^ T t ^ ) d Ω ^ {\displaystyle x_{a}({\hat {t}})={\frac {1}{2\pi ^{M}}}\int _{-\infty }^{+\infty }\!X({\hat {\Omega }})e^{(j{\hat {\Omega }}^{T}{\hat {t}})}\,\mathrm {d} {\hat {\Omega }}} The cap symbol ^ indicates that the operation is performed on vectors. The Fourier transform of the sampled signal is observed to be a periodic extension of the continuous time Fourier transform of the signal. This is mathematically represented as: X ( ω ) = 1 | d e t ( V ) | ∑ k X a ( Ω ^ − U k ) {\displaystyle X(\omega )={\frac {1}{|det(V)|}}\sum _{k}\!X_{a}({\hat {\Omega }}-Uk)} where ω = V ~ Ω {\displaystyle \omega ={\tilde {V}}\Omega } and U = 2 π V ~ {\displaystyle U=2\pi {\tilde {V}}} is the periodicity matrix where ~ denotes matrix transposition. Thus sampling in the spatial domain results in periodicity in the Fourier domain. === Aliasing === A band limited signal may be periodically replicated in many ways. If the replication results in an overlap between replicated regions, the signal suffers from aliasing. Under such conditions, a continuous time signal cannot be perfectly recovered from its samples. Thus in order to ensure perfect recovery of the continuous signal, there must be zero overlap multidimensional sampling of the replicated regions in the transformed domain. As in the case of 1-dimensional signals, aliasing can be prevented if the continuous time signal is sampled at an adequate sufficiently high rate. === Sampling density === It is a measure of the number of samples per unit area. It is defined as: S . D = 1 | d e t ( V ) | = | d e t ( U ) | 4 π 2 {\displaystyle S.D={\frac {1}{|det(V)|}}={\frac {|det(U)|}{4\pi ^{2}}}} . The minimum number of samples per unit area required to completely recover the continuous time signal is termed as optimal sampling density. In applications where memory or processing time are limited, emphasis must be given to minimizing the number of samples required to represent the signal completely. == Existing approaches == For a bandlimited waveform, there are infinitely many ways the signal can be sampled without producing aliases in the Fourier domain. But only two strategies are commonly used: rectangular sampling and hexagonal sampling. === Rectangular and Hexagonal sampling === In rectangular sampling, a 2-dimensional signal, for example, is sampled according to the following V matrix: V r e c t = [ T 1 0 0 T 2 ] {\displaystyle V_{rect}={\begin{bmatrix}T1&0\\0&T2\end{bmatrix}}} where T1 and T2 are the sampling periods along the horizontal and vertical direction respectively. In hexagonal sampling, the V matrix assumes the following general form: V h e x = [ T 1 T 1 − T 2 T 2 ] {\displaystyle V_{hex}={\begin{bmatrix}T1&T1\\-T2&T2\end{bmatrix}}} The difference in the efficiency of the two schemes is highlighted using a bandlimited signal with a circular region of support of radius R. The circle can be inscribed in a square of length 2R or a regular hexagon of length 2 R 3 {\displaystyle {\frac {2R}{\sqrt {3}}}} . Consequently, the region of support is now transformed into a square and a hexagon respectively. If these regions are periodically replicated in the frequency domain such that there is zero overlap between any two regions, then by periodically replicating the square region of support, we effectively sample the continuous signal on a rectangular lattice. Similarly periodic replication of the hexagonal region of support maps to sampling the continuous signal on a hexagonal lattice. From U, the periodicity matrix, we can calculate the optimal sampling density for both the rectangular and hexagonal schemes. It is found that in order to completely recover the circularly band-limited signal, the hexagonal sampling scheme requires 13.4% fewer samples than the rectangular sampling scheme. The reduction may appear to be of little significance for a 2-dimensional signal. But as the dimensionality of the signal increases, the efficiency of the hexagonal sampling scheme will become far more evident. For instance, the reduction achieved for an 8-dimensional signal is 93.8%. To highlight the importance of the obtained result [2], try and visualize an image as a collection of infinite number of samples. The primary entity responsible for vision, i.e. the photoreceptors (rods and cones) are present on the retina of all mammals. These cells are not arranged in rows and columns. By adapting a hexagonal sampling scheme, our eyes are able to process images much more efficiently. The importance of hexagonal sampling lies in the fact that the photoreceptors of the human vision system lie on a hexagonal sampling lattice and, thus, perform hexagonal sampling.[3] In fact, it can be shown that the hexagonal sampling scheme is the optimal sampling scheme for a circularly band-limited signal. == Applications == === Aliasing effects minimized by the use of optimal sampling grids === Recent advances in the CCD technology has made hexagonal sampling feasible for real life applications. Historically, because of technology constraints, detector arrays were implemented only on 2-dimensional rectangular sampling lattices with rectangular shape detectors. But the super [CCD] detector introduced by Fuji has an octagonal shaped pixel in a hexagonal grid. Theoretically, the performance of the detector was greatly increased by introducing an octagonal pixel. The number of pixels required to represent the sample was reduced and there was significant improvement in the Signal-to-Noise Ratio (SNR) when compared with that of a rectangular pixel. But the drawback of using hexagonal pixels is that the associated fill factor will be less than 82%. An alternative method would be to interpolate hexagonal pixels in such a manner that we ultimately end up with a rectangular grid. The Spot 5 satellite incorporates a

    Read more →
  • Dental AI

    Dental AI

    Dental artificial intelligence (Dental AI) refers to the application of artificial intelligence (AI) and machine-learning methods to oral healthcare data. These systems can be used to find patterns or make predictions that can aid in diagnosis, treatment, patient communication, or practice management. == History and development == Research into AI for dentistry dates to the 1990s and 2000s, alongside early CAD/CAM and image-analysis work in dental radiology. Recent developments in deep learning, especially those involving computer vision, such as convolutional neural networks, trained on large image datasets, led to a rapid improvement in performance, as well as a move from prototype technology to productization suitable for use in dental chairs. Dental schools and continuing education programs started incorporating AI content in the 2020s. == Definition and core technologies == The dental AI software accomplishes this task by using various dental images and patient data. Dental images and data used by the dental AI software include bitewing and periapical X-rays, complete mouth X-rays, detailed 3D images, intraoral images, and the patient’s medical history. The dental AI software utilizes several core technologies in accomplishing its task of assisting the dentist. First, the dental AI software utilizes machine learning and deep learning using programs that can learn from examples. Such programs are referred to as convolutional neural network (CNN) and can detect cavities and identify bone changes related to gum disease. The dental AI software utilizes computer vision, which enables the AI software to identify and quantify important features in images and data, whether they are 2D images or 3D images. Natural language processing (NLP) is used for the AI software to understand written text and can automatically generate dental notes and communicate with the patient. Furthermore, the dental AI software utilizes predictive analytics to identify patients that are more prone to dental complications and can suggest the best intervals for checkups or future dental procedures. == Applications in dentistry == Reported clinical and operational applications include diagnostic assistance for caries and periodontal disease, treatment planning assistance, patient education overlays, quality assurance, curriculum assistance for dental education, and claims documentation. Systematic reviews continue to find image-based applications such as caries detection with some variability in study design and a need for prospective validation. == Academic research and clinical validation == Several peer-reviewed studies have measured the effectiveness of AI for applications such as interproximal caries detection and periodontal bone level assessment, showing improvements over unaided readings with a focus on bias within the dataset. The Dental AI Council found variability among clinicians for diagnosis and treatment planning, suggesting the use of a standard tool as an assist. == Industry adoption == Multiple vendors offer FDA-cleared chairside AI for dental imaging: Pearl — Received U.S. FDA 510(k) clearance for its real-time radiologic aid (“Second Opinion”) in 2022 (2D), with subsequent clearances including pediatric and CBCT (“Second Opinion 3D”). TIME gave “Second Opinion” a special mention on its Best Inventions of 2022 list. Overjet — FDA-cleared for bone-level quantification and detection/outline of caries and calculus (e.g., K210187), with additional clearances expanding capabilities. VideaHealth — Received an FDA 510(k) covering 30+ detections across common dental findings (K232384), including indications for patients ages 3 and up; trade coverage has described elements of this as the first pediatric dental-AI clearance. == Regulations == In the U.S., AI-enabled dental imaging software is generally reviewed via the FDA’s 510(k) pathway. The FDA maintains a public AI-Enabled Medical Devices List, which includes numerous medical-imaging AI tools (including dental). Specific dental clearances include Overjet (K210187), VideaHealth (K232384), and Pearl entries such as “Second Opinion 3D” (K243989).

    Read more →
  • CrocBITE

    CrocBITE

    CrocBITE (currently CrocAttack) was an online database of wild crocodilian attacks reported on humans in the world. The non-profit online research tool helped to scientifically analyze crocodilian behavior via complex models. Users were encouraged to feed information in a crowdsourcing manner. This website excludes captive crocodilian attacks, as well as non-fatal bites on professional handlers, rangers, staff, or researchers, and crocodilian attacks on pets and livestock, because its primary goal is to analyze natural human-crocodilian conflict in the wild for conservation and management purposes, and that these incidents do are not considered indicative of natural species behavior or typical human-wildlife conflict, as well as not providing enough useful data and helping researchers understand wild population behavior or typical human-wildlife conflict dynamics and helps create safety strategies for people living or working near wild crocodilians, rather than tracking workplace accidents in zoos or farms. While fatal incidents involving handlers are sometimes included on the website, typical captive incidents (such as handlers being bitten by them in zoos) are excluded because they are considered manageable professional risks rather than general public safety threats. == About == The online database was established in 2013 (2013) by Dr Adam Britton, a researcher at Charles Darwin University, his student Brandon Sideleau and Erin Britton. It was a compilation of government records, individual reports, registered contributors and historical data. Dr Simon Pooley, Junior Research fellow, Imperial College London joined hands to further the studies. The collaboration culminated when Dr Pooley met Dr Britton at the IUCN Crocodile Specialist Group, in Louisiana in 2014. The program received funds from Economic and Social Research Council, United Kingdom to the tune of A$30,000 and unspecified resourced plus amount from Big Gecko Crocodilian Research, Crocodillian.com and Charles Darwin University. The research yielded pertinent observations that provide inside into crocodile attacks. It was observed that most attacks on humans occur from bites of Saltwater crocodile as against the popular understanding of Nile crocodiles taking the top spot. This is not, however, believed to be the actual case, as most attacks by the Nile crocodile are believed to go unreported or only reported on a local level. The broad category of Nile crocodile attacks were segmented into West African crocodile and Crocodylus niloticus (the Nile Crocodile) species to get a clear understanding of their respective attack zones. The objective was that the information would be used by communities and conservation managers to help inform and educate people about how to keep safe. The information was vital for Australia and Africa where such attacks are more likely than in other parts of the world. This was the only database of its kind with such comprehensive collection of information made available online. The database is no longer online, and its founder Adam Britton is in custody having pleaded guilty to charges of bestiality on September 25, 2023. It has been rebranded and renamed CrocAttack, and serves as a updated database focusing on human-crocodilian conflict and records over 8,500 incidents from the past decades.

    Read more →
  • Secure state

    Secure state

    A secure state is an information systems security term to describe where entities in a computer system are divided into subjects and objects, and it can be formally proven that each state transition preserves security by moving from one secure state to another secure state. Thereby it can be inductively proven that the system is secure. As defined in the Bell–LaPadula model, the secure state is built on the concept of a state machine with a set of allowable states in a system. The transition from one state to another state is defined by transition functions. A system state is defined to be "secure" if the only permitted access modes of subjects to objects are in accordance with a security policy.

    Read more →
  • WebGPU Shading Language

    WebGPU Shading Language

    WebGPU Shading Language (WGSL, internet media type: text/wgsl) is a high-level shading language and the normative shader language for the WebGPU API on the web. WGSL's syntax is influenced by Rust and is designed with strong static validation, explicit resource binding, and portability in mind for secure execution in browsers. In web contexts, WebGPU implementations accept WGSL source and perform compilation to platform-specific intermediate forms (for example, to SPIR‑V, DXIL, or MSL via the user agent), but such backends are not exposed to web content. == History and background == Graphics on the web historically used WebGL, with shaders written in GLSL ES. As applications demanded more modern GPU features and finer control over compute and graphics pipelines, the W3C's GPU for the Web Community Group and Working Group created WebGPU and its companion shading language, WGSL, to provide a secure, portable model suitable for the web platform. WGSL was developed to be human-readable, avoid undefined behavior common in legacy shading languages, and align closely with WebGPU's resource and validation model. == Design goals == WGSL's design emphasizes: Safety and determinism suitable for web security constraints (extensive static validation and well-defined semantics). Portability across diverse GPU backends via an abstract resource model shared with WebGPU. Readability and explicitness (no preprocessor, minimal implicit conversions, explicit address spaces and bindings). Alignment with modern GPU features (compute, storage buffers, textures, atomics) while retaining a familiar C/Rust-like syntax. == Language overview == === Types and values === Core scalar types include bool, i32, u32, and f32. Vectors (e.g., vec2, vec3, vec4) and matrices (up to 4×4) are available for floating-point element types. Optional f16 (half precision) may be enabled via a WebGPU feature; availability is implementation-dependent. Atomic types (atomic, atomic) support limited atomic operations in qualified address spaces. === Variables and address spaces === Variables are declared with let (immutable), var (mutable), or const (compile-time constant). Storage classes (address spaces) include function, private, workgroup, uniform, and storage with read or read_write access as applicable. WGSL defines explicit layout and alignment rules; attributes such as @align, @size, and @stride control data layout for buffer interoperability. === Functions and control flow === Functions use explicit parameter and return types. Control flow includes if, switch, for, while, and loop constructs, with break/continue. Recursion is disallowed; entry-point call graphs must be acyclic. === Entry points and attributes === Shaders define stage entry points with @vertex, @fragment, or @compute. Attributes annotate bindings and interfaces, including @group, @binding (resource binding), @location (user-defined I/O), @builtin (stage built-ins such as position or global_invocation_id), @interpolate, and @workgroup_size. === Resources === WGSL exposes buffers (uniform, storage), textures (sampled, storage, and multisampled variants), and samplers (filtering/non-filtering/comparison). The binding model is explicit via descriptor sets called groups and bindings, matching WebGPU's pipeline layout model. == Compilation and validation == Browsers compile WGSL to platform-appropriate representations and native driver formats; the specific compilation pipeline is not observable by web content. WGSL source undergoes strict parsing and static validation, and WebGPU enforces robust resource access rules to avoid out-of-bounds memory hazards, contributing to predictable behavior across implementations. == Shader stages == WGSL supports three pipeline stages: vertex, fragment, and compute. === Vertex shaders === Vertex shaders transform per-vertex inputs and produce values for rasterization, including a clip-space position written to the position builtin. ==== Example ==== === Fragment shaders === Fragment shaders run per-fragment and compute color (and optionally depth) outputs written to color attachments. ==== Example ==== If half-precision (vec4h, shorthand for vec4) is desired, the code must be prefaced with a enable f16; statement. === Compute shaders === Compute shaders run in workgroups and are used for general-purpose GPU computations. ==== Example ==== == Differences from GLSL and HLSL == Compared with legacy shading languages, WGSL: Omits a preprocessor and requires explicit types and conversions. Uses explicit address spaces and binding annotations aligned with WebGPU's model. Enforces strict validation to avoid undefined behavior common in other shading languages. Defines a portable, web-focused feature set; 16-bit types and other features are opt-in and may depend on device capabilities.

    Read more →
  • Self-supervised learning

    Self-supervised learning

    Self-supervised learning (SSL) is a paradigm in machine learning where a model is trained on a task using the data itself to generate supervisory signals, rather than relying on externally-provided labels. In the context of neural networks, self-supervised learning aims to leverage inherent structures or relationships within the input data to create meaningful training signals. SSL tasks are designed so that solving them requires capturing essential features or relationships in the data. The input data is typically augmented or transformed in a way that creates pairs of related samples, where one sample serves as the input, and the other is used to formulate the supervisory signal. This augmentation can involve introducing noise, cropping, rotation, or other transformations. Self-supervised learning more closely imitates the way humans learn to classify objects. During SSL, the model learns in two steps. First, the task is solved based on an auxiliary or pretext classification task using pseudo-labels, which help to initialize the model parameters. Next, the actual task is performed with supervised or unsupervised learning. Self-supervised learning has produced promising results in recent years, and has found practical application in fields such as audio processing, and is being used by Facebook and others for speech recognition. == Pseudo-labels == Pseudo-labels are automatically generated labels that a model assigns to unlabeled data based on its own predictions. They are widely used in self-supervised and semi-supervised learning, where ground-truth annotations are limited or unavailable. By treating predicted labels as surrogate ground truth, learning algorithms can make use of large quantities of unlabeled data in the training process. Pseudo-labeling also plays an important role in systems that must adapt to concept drift, where the statistical properties of the data change over time. In these scenarios, the model may detect that an incoming instance deviates from previously learned behavior. The system then generates a classification result for that instance, and this predicted class is used as a pseudo-label for updating or retraining model components that are becoming outdated. This approach enables continuous adaptation in dynamic environments without requiring manual annotation. In many adaptive learning pipelines, pseudo-labels are chosen when the classifier produces sufficiently confident predictions, reducing the risk of propagating errors. These pseudo-labeled instances are then incorporated into training to refresh or evolve the model's understanding of emerging data patterns, particularly when existing components show signs of “aging” due to drift or distributional shifts. This strategy reduces reliance on manual labeling while helping maintain long-term model performance. == Types == === Autoassociative self-supervised learning === Autoassociative self-supervised learning is a specific category of self-supervised learning where a neural network is trained to reproduce or reconstruct its own input data. In other words, the model is tasked with learning a representation of the data that captures its essential features or structure, allowing it to regenerate the original input. The term "autoassociative" comes from the fact that the model is essentially associating the input data with itself. This is often achieved using autoencoders, which are a type of neural network architecture used for representation learning. Autoencoders consist of an encoder network that maps the input data to a lower-dimensional representation (latent space), and a decoder network that reconstructs the input from this representation. The training process involves presenting the model with input data and requiring it to reconstruct the same data as closely as possible. The loss function used during training typically penalizes the difference between the original input and the reconstructed output (e.g. mean squared error). By minimizing this reconstruction error, the autoencoder learns a meaningful representation of the data in its latent space. === Contrastive self-supervised learning === For a binary classification task, training data can be divided into positive examples and negative examples. Positive examples are those that match the target. For example, if training a classifier to identify birds, the positive training data would include images that contain birds. Negative examples would be images that do not. Contrastive self-supervised learning uses both positive and negative examples. The loss function in contrastive learning is used to minimize the distance between positive sample pairs, while maximizing the distance between negative sample pairs. An early example uses a pair of 1-dimensional convolutional neural networks to process a pair of images and maximize their agreement. Contrastive Language-Image Pre-training (CLIP) allows joint pretraining of a text encoder and an image encoder, such that a matching image-text pair have image encoding vector and text encoding vector that span a small angle (having a large cosine similarity). InfoNCE (Noise-Contrastive Estimation) is a method to optimize two models jointly, based on Noise Contrastive Estimation (NCE). Given a set X = { x 1 , … x N } {\displaystyle X=\left\{x_{1},\ldots x_{N}\right\}} of N {\displaystyle N} random samples containing one positive sample from p ( x t + k ∣ c t ) {\displaystyle p\left(x_{t+k}\mid c_{t}\right)} and N − 1 {\displaystyle N-1} negative samples from the 'proposal' distribution p ( x t + k ) {\displaystyle p\left(x_{t+k}\right)} , it minimizes the following loss function: L N = − E X [ log ⁡ f k ( x t + k , c t ) ∑ x j ∈ X f k ( x j , c t ) ] {\displaystyle {\mathcal {L}}_{\mathrm {N} }=-\mathbb {E} _{X}\left[\log {\frac {f_{k}\left(x_{t+k},c_{t}\right)}{\sum _{x_{j}\in X}f_{k}\left(x_{j},c_{t}\right)}}\right]} === Non-contrastive self-supervised learning === Non-contrastive self-supervised learning (NCSSL) uses only positive examples. Counterintuitively, NCSSL converges on a useful local minimum rather than reaching a trivial solution, with zero loss. For the example of binary classification, it would trivially learn to classify each example as positive. Effective NCSSL requires an extra predictor on the online side that does not back-propagate on the target side. === Joint-Embedding and Predictive Architectures === A major class of self-supervised learning moves beyond contrastive pairs, instead maximizing the agreement between views while preventing collapse through statistical constraints. Rooted in Deep Canonical Correlation Analysis (Deep CCA), this approach includes Joint-Embedding Architectures (JEA) like Barlow Twins and VICReg, which enforce covariance constraints to learn invariant representations without negative sampling. Deep Latent Variable Path Modelling (DLVPM) generalizes this to multimodal systems, using path models to enforce correlation and orthogonality across diverse data types. In 2022 Yann LeCun introduced Joint-Embedding Predictive Architectures (JEPA) as a step towards decision making, reasoning, and autonomous human intelligence in machines, including self-improvement through autonomous learning. Founded in representation learning, LeCun included the concept of a “world model” in JEPA which aims to enable machines to replicate human intellect by providing machines with a concept for the world in which they exist. Unlike autoencoders, JEPAs operate entirely in latent space, avoiding pixel-level noise to focus on semantic structure. Rather than just learning invariance, JEPAs learn by predicting masked latent representations from visible context. JEPA has been applied to domains such as image analysis, audio processing, and motion in images and video. == Comparison with other forms of machine learning == SSL belongs to supervised learning methods insofar as the goal is to generate a classified output from the input. At the same time, however, it does not require the explicit use of labeled input-output pairs. Instead, correlations, metadata embedded in the data, or domain knowledge present in the input are implicitly and autonomously extracted from the data. These supervisory signals, extracted from the data, can then be used for training. SSL is similar to unsupervised learning in that it does not require labels in the sample data. Unlike unsupervised learning, however, learning is not done using inherent data structures. Semi-supervised learning combines supervised and unsupervised learning, requiring only a small portion of the learning data be labeled. In transfer learning, a model designed for one task is reused on a different task. Training an autoencoder intrinsically constitutes a self-supervised process, because the output pattern needs to become an optimal reconstruction of the input pattern itself. However, in current jargon, the term 'self-supervised' often refers to tasks based on a pretext-task training setup

    Read more →
  • Concurrency control

    Concurrency control

    In information technology and computer science, especially in the fields of computer programming, operating systems, multiprocessors, and databases, concurrency control ensures that correct results for concurrent operations are generated, while getting those results as quickly as possible. Computer systems, both software and hardware, consist of modules, or components. Each component is designed to operate correctly, i.e., to obey or to meet certain consistency rules. When components that operate concurrently interact by messaging or by sharing accessed data (in memory or storage), a certain component's consistency may be violated by another component. The general area of concurrency control provides rules, methods, design methodologies, and theories to maintain the consistency of components operating concurrently while interacting, and thus the consistency and correctness of the whole system. Introducing concurrency control into a system means applying operation constraints which typically result in some performance reduction. Operation consistency and correctness should be achieved with as good as possible efficiency, without reducing performance below reasonable levels. Concurrency control can require significant additional complexity and overhead in a concurrent algorithm compared to the simpler sequential algorithm. For example, a failure in concurrency control can result in data corruption from torn read or write operations. == Concurrency control in databases == Comments: This section is applicable to all transactional systems, i.e., to all systems that use database transactions (atomic transactions; e.g., transactional objects in Systems management and in networks of smartphones which typically implement private, dedicated database systems), not only general-purpose database management systems (DBMSs). DBMSs need to deal also with concurrency control issues not typical just to database transactions but rather to operating systems in general. These issues (e.g., see Concurrency control in operating systems below) are out of the scope of this section. Concurrency control in Database management systems (DBMS; e.g., Bernstein et al. 1987, Weikum and Vossen 2001), other transactional objects, and related distributed applications (e.g., Grid computing and Cloud computing) ensures that database transactions are performed concurrently without violating the data integrity of the respective databases. Thus concurrency control is an essential element for correctness in any system where two database transactions or more, executed with time overlap, can access the same data, e.g., virtually in any general-purpose database system. Consequently, a vast body of related research has been accumulated since database systems emerged in the early 1970s. A well established concurrency control theory for database systems is outlined in the references mentioned above: serializability theory, which allows to effectively design and analyze concurrency control methods and mechanisms. An alternative theory for concurrency control of atomic transactions over abstract data types is presented in (Lynch et al. 1993), and not utilized below. This theory is more refined, complex, with a wider scope, and has been less utilized in the Database literature than the classical theory above. Each theory has its pros and cons, emphasis and insight. To some extent they are complementary, and their merging may be useful. To ensure correctness, a DBMS usually guarantees that only serializable transaction schedules are generated, unless serializability is intentionally relaxed to increase performance, but only in cases where application correctness is not harmed. For maintaining correctness in cases of failed (aborted) transactions (which can always happen for many reasons) schedules also need to have the recoverability (from abort) property. A DBMS also guarantees that no effect of committed transactions is lost, and no effect of aborted (rolled back) transactions remains in the related database. Overall transaction characterization is usually summarized by the ACID rules below. As databases have become distributed, or needed to cooperate in distributed environments (e.g., Federated databases in the early 1990, and Cloud computing currently), the effective distribution of concurrency control mechanisms has received special attention. === Database transaction and the ACID rules === The concept of a database transaction (or atomic transaction) has evolved in order to enable both a well understood database system behavior in a faulty environment where crashes can happen any time, and recovery from a crash to a well understood database state. A database transaction is a unit of work, typically encapsulating a number of operations over a database (e.g., reading a database object, writing, acquiring lock, etc.), an abstraction supported in database and also other systems. Each transaction has well defined boundaries in terms of which program/code executions are included in that transaction (determined by the transaction's programmer via special transaction commands). Every database transaction obeys the following rules (by support in the database system; i.e., a database system is designed to guarantee them for the transactions it runs): Atomicity - Either the effects of all or none of its operations remain ("all or nothing" semantics) when a transaction is completed (committed or aborted respectively). In other words, to the outside world a committed transaction appears (by its effects on the database) to be indivisible (atomic), and an aborted transaction does not affect the database at all. Either all the operations are done or none of them are. Consistency - Every transaction must leave the database in a consistent (correct) state, i.e., maintain the predetermined integrity rules of the database (constraints upon and among the database's objects). A transaction must transform a database from one consistent state to another consistent state (however, it is the responsibility of the transaction's programmer to make sure that the transaction itself is correct, i.e., performs correctly what it intends to perform (from the application's point of view) while the predefined integrity rules are enforced by the DBMS). Thus since a database can be normally changed only by transactions, all the database's states are consistent. Isolation - Transactions cannot interfere with each other (as an end result of their executions). Moreover, usually (depending on concurrency control method) the effects of an incomplete transaction are not even visible to another transaction. Providing isolation is the main goal of concurrency control. Durability - Effects of successful (committed) transactions must persist through crashes (typically by recording the transaction's effects and its commit event in a non-volatile memory). The concept of atomic transaction has been extended during the years to what has become Business transactions which actually implement types of Workflow and are not atomic. However also such enhanced transactions typically utilize atomic transactions as components. === Why is concurrency control needed? === If transactions are executed serially, i.e., sequentially with no overlap in time, no transaction concurrency exists. However, if concurrent transactions with interleaving operations are allowed in an uncontrolled manner, some unexpected, undesirable results may occur, such as: The lost update problem: A second transaction writes a second value of a data-item (datum) on top of a first value written by a first concurrent transaction, and the first value is lost to other transactions running concurrently which need, by their precedence, to read the first value. The transactions that have read the wrong value end with incorrect results. The dirty read problem: Transactions read a value written by a transaction that has been later aborted. This value disappears from the database upon abort, and should not have been read by any transaction ("dirty read"). The reading transactions end with incorrect results. The incorrect summary problem: While one transaction takes a summary over the values of all the instances of a repeated data-item, a second transaction updates some instances of that data-item. The resulting summary does not reflect a correct result for any (usually needed for correctness) precedence order between the two transactions (if one is executed before the other), but rather some random result, depending on the timing of the updates, and whether certain update results have been included in the summary or not. Most high-performance transactional systems need to run transactions concurrently to meet their performance requirements. Thus, without concurrency control such systems can neither provide correct results nor maintain their databases consistently. === Concurrency control mechanisms === ==== Categories ==== The main categories of concurrency control mechanis

    Read more →
  • Information schema

    Information schema

    In relational databases, the information schema (information_schema) is an ANSI-standard set of read-only views that provide information about all of the tables, views, columns, and procedures in a database. It can be used as a source of the information that some databases make available through non-standard commands, such as: the SHOW command of MySQL the DESCRIBE command of Oracle's SQLPlus the \d command in psql (PostgreSQL's default command-line program). => SELECT count(table_name) FROM information_schema.tables; count ------- 99 (1 row) => SELECT column_name, data_type, column_default, is_nullable FROM information_schema.columns WHERE table_name='alpha'; column_name | data_type | column_default | is_nullable -------------+-----------+----------------+------------- foo | integer | | YES bar | character | | YES (2 rows) => SELECT FROM information_schema.information_schema_catalog_name; catalog_name -------------- johnd (1 row) == Implementation == As a notable exception among major database systems, Oracle does not as of 2015 implement the information schema. An open-source project exists to address this. RDBMSs that support information_schema include: Amazon Redshift Apache Hive Microsoft SQL Server MonetDB Snowflake MySQL PostgreSQL H2 Database HSQLDB InterSystems Caché MariaDB SingleStore (formerly MemSQL) Mimer SQL Snowflake Trino Presto CrateDB ClickHouse CockroachDB Kinetica DB TiDB RDBMSs that do not support information_schema include: Apache Derby Apache Ignite Firebird Microsoft Access IBM Informix Ingres IBM Db2 Oracle Database SAP HANA SQLite Sybase ASE Sybase SQL Anywhere Teradata Vertica

    Read more →