Big data

Big data

Big data primarily refers to data sets that are too large or complex to be dealt with by traditional data-processing software. Data with many entries (rows) offers greater statistical power, while data with higher complexity (more attributes or columns) may lead to a higher false discovery rate. Big data analysis challenges include capturing data, data storage, data analysis, search, sharing, transfer, visualization, querying, updating, information privacy, and data sources. Big data was originally associated with three key concepts: volume, variety, and velocity. The analysis of big data that have only volume, velocity, and variety can pose challenges in sampling. A fourth concept, veracity, which refers to the level of reliability of data, was thus added. Without sufficient investment in expertise to ensure big data veracity, the volume and variety of data can produce costs and risks that exceed an organization's capacity to create and capture value from big data. Current usage of the term big data tends to refer to the use of predictive analytics, user behavior analytics, or certain other advanced data analytics methods that extract value from big data, and seldom to a particular size of data set. "There is little doubt that the quantities of data now available are indeed large, but that's not the most relevant characteristic of this new data ecosystem." Analysis of data sets can find new correlations to "spot business trends, prevent diseases, combat crime and so on". Scientists, business executives, medical practitioners, advertising and governments alike regularly meet difficulties with large datasets in areas including Internet searches, fintech, healthcare analytics, geographic information systems, urban informatics, and business informatics. Scientists encounter limitations in e-Science work, including meteorology, genomics, connectomics, complex physics simulations, biology, and environmental research. The size and number of available data sets have grown rapidly as data is collected by devices such as mobile devices, cheap and numerous information-sensing Internet of things devices, aerial (remote sensing) equipment, software logs, cameras, microphones, radio-frequency identification (RFID) readers and wireless sensor networks. The world's technological per-capita capacity to store information has roughly doubled every 40 months since the 1980s; as of 2012, every day 2.5 exabytes (2.17×260 bytes) of data are generated. Based on an IDC report prediction, the global data volume was predicted to grow exponentially from 4.4 zettabytes to 44 zettabytes between 2013 and 2020. By 2025, IDC predicts there will be 163 zettabytes of data. According to IDC, global spending on big data and business analytics (BDA) solutions is estimated to reach $215.7 billion in 2021. Statista reported that the global big data market is forecasted to grow to $103 billion by 2027. In 2011 McKinsey & Company reported, if US healthcare were to use big data creatively and effectively to drive efficiency and quality, the sector could create more than $300 billion in value every year. In the developed economies of Europe, government administrators could save more than €100 billion ($149 billion) in operational efficiency improvements alone by using big data. And users of services enabled by personal-location data could capture $600 billion in consumer surplus. One question for large enterprises is determining who should own big-data initiatives that affect the entire organization. Relational database management systems and desktop statistical software packages used to visualize data often have difficulty processing and analyzing big data. The processing and analysis of big data may require "massively parallel software running on tens, hundreds, or even thousands of servers". What qualifies as "big data" varies depending on the capabilities of those analyzing it and their tools. Furthermore, expanding capabilities make big data a moving target. "For some organizations, facing hundreds of gigabytes of data for the first time may trigger a need to reconsider data management options. For others, it may take tens or hundreds of terabytes before data size becomes a significant consideration." == Definition == The term big data has been in use since the 1990s, with some giving credit to John Mashey for popularizing the term. Big data usually includes data sets with sizes beyond the ability of commonly used software tools to capture, curate, manage, and process data within a tolerable elapsed time. Big data philosophy encompasses unstructured, semi-structured and structured data; however, the main focus is on unstructured data. Big data "size" is a constantly moving target; as of 2012 ranging from a few dozen terabytes to many zettabytes of data. Big data requires a set of techniques and technologies with new forms of integration to reveal insights from datasets that are diverse, complex, and of a massive scale. Variability is often included as an additional quality of big data. A 2018 definition states "Big data is where parallel computing tools are needed to handle data", and notes, "This represents a distinct and clearly defined change in the computer science used, via parallel programming theories, and losses of some of the guarantees and capabilities made by Codd's relational model." In a comparative study of big datasets, Kitchin and McArdle found that none of the commonly considered characteristics of big data appear consistently across all of the analyzed cases. For this reason, other studies identified the redefinition of power dynamics in knowledge discovery as the defining trait. Instead of focusing on the intrinsic characteristics of big data, this alternative perspective pushes forward a relational understanding of the object claiming that what matters is the way in which data is collected, stored, made available and analyzed. === Big data vs. business intelligence === The growing maturity of the concept more starkly delineates the difference between "big data" and "business intelligence": Business intelligence uses applied mathematics tools and descriptive statistics with data with high information density to measure things, detect trends, etc. Big data uses mathematical analysis, optimization, inductive statistics, and concepts from nonlinear system identification to infer laws (regressions, nonlinear relationships, and causal effects) from large sets of data with low information density to reveal relationships and dependencies, or to perform predictions of outcomes and behaviors. == Characteristics == Big data can be described by the following characteristics: Volume The quantity of generated and stored data. The size of the data determines the value and potential insight, and whether it can be considered big data or not. The size of big data is usually larger than terabytes and petabytes. Variety The type and nature of the data. Earlier technologies like RDBMSs were capable to handle structured data efficiently and effectively. However, the change in type and nature from structured to semi-structured or unstructured challenged the existing tools and technologies. Big data technologies evolved with the prime intention to capture, store, and process the semi-structured and unstructured (variety) data generated with high speed (velocity), and huge in size (volume). Later, these tools and technologies were explored and used for handling structured data also but preferable for storage. Eventually, the processing of structured data was still kept as optional, either using big data or traditional RDBMSs. This helps in analyzing data towards effective usage of the hidden insights exposed from the data collected via social media, log files, sensors, etc. Big data draws from text, images, audio, video; plus it completes missing pieces through data fusion. Velocity The speed at which the data is generated and processed to meet the demands and challenges that lie in the path of growth and development. Big data is often available in real-time. Compared to small data, big data is produced more continually. Two kinds of velocity related to big data are the frequency of generation and the frequency of handling, recording, and publishing. Veracity The truthfulness or reliability of the data, which refers to the data quality and the data value. Big data must not only be large in size, but also must be reliable in order to achieve value in the analysis of it. The data quality of captured data can vary greatly, affecting an accurate analysis. Value The worth in information that can be achieved by the processing and analysis of large datasets. Value also can be measured by an assessment of the other qualities of big data. Value may also represent the profitability of information that is retrieved from the analysis of big data. Variability The characteristic of the changing formats, structure, or sources of big data. Big data can include structured, unstructured,

Database application

A database application is a computer program whose primary purpose is retrieving information from a computerized database. From here, information can be inserted, modified or deleted which is subsequently conveyed back into the database. Early examples of database applications were accounting systems and airline reservations systems, such as SABRE, developed starting in 1957. A characteristic of modern database applications is that they facilitate simultaneous updates and queries from multiple users. Systems in the 1970s might have accomplished this by having each user in front of a 3270 terminal to a mainframe computer. By the mid-1980s it was becoming more common to give each user a personal computer and have a program running on that PC that is connected to a database server. Information would be pulled from the database, transmitted over a network, and then arranged, graphed, or otherwise formatted by the program running on the PC. Starting in the mid-1990s it became more common to build database applications with a Web interface. Rather than develop custom software to run on a user's PC, the user would use the same Web browser program for every application. A database application with a Web interface had the advantage that it could be used on devices of different sizes, with different hardware, and with different operating systems. Examples of early database applications with Web interfaces include amazon.com, which used the Oracle relational database management system, the photo.net online community, whose implementation on top of Oracle was described in the book Database-Backed Web Sites (Ziff-Davis Press; May 1997), and eBay, also running Oracle. Electronic medical records are referred to on emrexperts.com, in December 2010, as "a software database application". A 2005 O'Reilly book uses the term in its title: Database Applications and the Web. Some of the most complex database applications remain accounting systems, such as SAP, which may contain thousands of tables in only a single module. Many of today's most widely used computer systems are database applications, for example, Facebook, which was built on top of MySQL. The etymology of the phrase "database application" comes from the practice of dividing computer software into systems programs, such as the operating system, compilers, the file system, and tools such as the database management system, and application programs, such as a payroll check processor. On a standard PC running Microsoft Windows, for example, the Windows operating system contains all of the systems programs while games, word processors, spreadsheet programs, photo editing programs, etc. would be application programs. As "application" is short for "application program", "database application" is short for "database application program". Not every program that uses a database would typically be considered a "database application". For example, many physics experiments, e.g., the Large Hadron Collider, generate massive data sets that programs subsequently analyze. The data sets constitute a "database", though they are not typically managed with a standard relational database management system. The computer programs that analyze the data are primarily developed to answer hypotheses, not to put information back into the database and therefore the overall program would not be called a "database application". == Examples of database applications == Amazon Student Data CNN eBay Facebook Fandango Filemaker (Mac OS) LibreOffice Base Microsoft Access Oracle relational database SAP (Systems, Applications & Products in Data Processing) Ticketmaster Wikipedia Yelp YouTube Google MySQL

Fuzzy Control Language

Fuzzy Control Language, or FCL, is a language for implementing fuzzy logic, especially fuzzy control. It was standardized by IEC 61131-7. It is a domain-specific programming language: it has no features unrelated to fuzzy logic, so it is impossible to even print "Hello, world!". Therefore, one does not write a program in FCL, but one may write part of it in FCL. == Example == RULE 0: IF (temperature IS cold) THEN (output IS low) RULE 1: IF (temperature IS very cold) THEN (output IS high) == Limitations == FCL is not an entirely complete fuzzy language, for instance, it does not support "hedges", which are adverbs that modify the set. For instance, the programmer cannot write: RULE 0: If (Temperature is VERY COLD) then (Output is VERY HIGH) However, the programmer can simply define new sets for "very cold" and "very high". FCL also lacks support for higher-order fuzzy sets, subsets, and so on. None of these features are essential to fuzzy control, although they may be nice to have.

Fuzzy markup language

Fuzzy Markup Language (FML) is a specific purpose markup language based on XML, used for describing the structure and behavior of a fuzzy system independently of the hardware architecture devoted to host and run it. == Overview == FML was designed and developed by Giovanni Acampora during his Ph.D. course in Computer Science, at University of Salerno, Italy, in 2004. The original idea inspired Giovanni Acampora to create FML was the necessity of creating a cooperative fuzzy-based framework aimed at automatically controlling a living environment characterized by a plethora of heterogeneous devices whose interactions were devoted to maximize the human comfort under energy saving constraints. This framework represented one of the first concrete examples of Ambient Intelligence. Beyond this pioneering application, the major advantage of using XML to describe a fuzzy system is hardware/software interoperability. Indeed, all that is needed to read an FML file is the appropriate schema for that file, and an FML parser. This markup approach makes it much easier to exchange fuzzy systems between software: for example, a machine learning application could extract fuzzy rules which could then be read directly into a fuzzy inference engine or uploaded into a fuzzy controller. Also, with technologies like XSLT, it is possible to compile the FML into the programming language of your choice, ready for embedding into whatever application you please. As stated by Mike Watts on his popular Computational Intelligence blog: "Although Acampora's motivation for developing FML seems to be to develop embedded fuzzy controllers for ambient intelligence applications, FML could be a real boon for developers of fuzzy rule extraction algorithms: from my own experience during my PhD, I know that having to design a file format and implement the appropriate parsers for rule extraction and fuzzy inference engines can be a real pain, taking as much time as implementing the rule extraction algorithm itself. I would much rather have used something like FML for my work." A complete overview of FML and related applications can be found in the book titled On the power of Fuzzy Markup Language edited by Giovanni Acampora, Chang-Shing Lee, Vincenzo Loia and Mei-Hui Wang, and published by Springer in the series Studies on Fuzziness and Soft Computing. == Syntax, grammar and hardware synthesis == FML allows fuzzy systems to be coded through a collection of correlated semantic tags capable of modeling the different components of a classical fuzzy controller such as knowledge base, rule base, fuzzy variables and fuzzy rules. Therefore, the FML tags used to build a fuzzy controller represent the set of lexemes used to create fuzzy expressions. In order to design a well-formed XML-based language, an FML context-free grammar is defined by means of a XML schema which defines name, type and attributes characterized each XML element. However, since an FML program represents only a static view of a fuzzy logic controller, XSLT is provided to change this static view to a computable version. Indeed, XSLTs modules are able to convert the FML-based fuzzy controller in a general purpose computer language using an XSL file containing the translation description. At this level, the control is executable for the hardware. In short, FML is essentially composed by three layers: XML in order to create a new markup language for fuzzy logic control; a XML Schema in order to define the legal building blocks; eXtensible Stylesheet Language Transformations (XSLT) in order to convert a fuzzy controller description into a specific programming language. === Syntax === FML syntax is composed of XML tags and attributes which describe the different components of a fuzzy logic controller listed below: fuzzy knowledge base; fuzzy rule base; inference engine fuzzification subsystem; defuzzification subsystem. In detail, the opening tag of each FML program is which represents the fuzzy controller under modeling. This tag has two attributes: name and ip. The first attribute permits to specify the name of fuzzy controller and ip is used to define the location of controller in a computer network. The fuzzy knowledge base is defined by means of the tag which maintains the set of fuzzy concepts used to model the fuzzy rule base. In order to define the fuzzy concept related controlled system, tag uses a set of nested tags: defines the fuzzy concept; defines a linguistic term describing the fuzzy concept; a set of tags defining a shape of fuzzy sets are related to fuzzy terms. The attributes of tag are: name, scale, domainLeft, domainRight, type and, for only an output, accumulation, defuzzifier and defaultValue. The name attribute defines the name of fuzzy concept, for instance, temperature; scale is used to define the scale used to measure the fuzzy concept, for instance, Celsius degree; domainLeft and domainRight are used to model the universe of discourse of fuzzy concept, that is, the set of real values related to fuzzy concept, for instance [0°,40°] in the case of Celsius degree; the position of fuzzy concept into rule (consequent part or antecedent part) is defined by type attribute (input/output); accumulation attribute defines the method of accumulation that is a method that permits the combination of results of a variable of each rule in a final result; defuzzifier attribute defines the method used to execute the conversion from a fuzzy set, obtained after aggregation process, into a numerical value to give it in output to system; defaultValue attribute defines a real value used only when no rule has fired for the variable at issue. As for tag , it uses two attributes: name used to identify the linguistic value associate with fuzzy concept and complement, a boolean attribute that defines, if it is true, it is necessary to consider the complement of membership function defined by given parameters. Fuzzy shape tags, used to complete the definition of fuzzy concept, are: Every shaping tag uses a set of attributes which defines the real outline of corresponding fuzzy set. The number of these attributes depends on the chosen fuzzy set shape. In order to make an example, consider the Tipper Inference System described in Mathworks Matlab Fuzzy Logic Toolbox Tutorial. This Mamdani system is used to regulate the tipping in, for example, a restaurant. It has got two variables in input (food and service) and one in output (tip). FML code for modeling part of knowledge base of this fuzzy system containing variables food and tip is shown below. A special tag that can furthermore be used to define a fuzzy shape is . This tag is used to customize fuzzy shape (custom shape). The custom shape modeling is performed via a set of tags that lists the extreme points of geometric area defining the custom fuzzy shape. Obviously, the attributes used in tag are x and y coordinates. As for rule base component, FML allows to define a set of rule bases, each one of them describes a different behavior of system. The root of each rule base is modeled by tag which defines a fuzzy rule set. The tag uses five attributes: name, type, activationMethod, andMethod and orMethod. Obviously, the name attribute uniquely identifies the rule base. The type attribute permits to specify the kind of fuzzy controller (Mamdani or TSK) respect to the rule base at issue. The activationMethod attribute defines the method used to implication process; the andMethod and orMethod attribute define, respectively, the and and or algorithm to use by default. In order to define the single rule the tag is used. The attributes used by the tag are: name, connector, operator and weight. The name attribute permits to identify the rule; connector is used to define the logical operator used to connect the different clauses in antecedent part (and/or); operator defines the algorithm to use for chosen connector; weight defines the importance of rule during inference engine step. The definition of antecedent and consequent rule part is obtained by using and tags. tag is used to model the fuzzy clauses in antecedent and consequent part. This tag use the attribute modifier to describe a modification to term used in the clause. The possible values for this attribute are: above, below, extremely, intensify, more or less, norm, not, plus, slightly, somewhat, very, none. To complete the definition of fuzzy clause the nested and tags have to be used. A sequence of tags realizes a fuzzy rule base. As example, consider a Mamdani rule composed by (food is rancid) OR (servi

For a Breath I Tarry

"For a Breath I Tarry" is a 1966 post-apocalyptic novelette by American writer Roger Zelazny, which was nominated for the Hugo Award for Best Novelette in 1967. Set in a future long after the self-extinction of humanity, the novelette recounts the tale of Frost, a sentient machine. Although humans have caused their own extinction, the sentient machines that they created continue the work of rebuilding a shattered Earth. Along the way, the story explores the differences between humanity and machines, the former experiencing the world qualitatively, while the latter doing so quantitatively. This difference is illustrated through philosophical conversations between Frost and another machine named Mordel. Frost's goal of becoming human, along with literary allusions, drives the plot and sets the tone of the novelette. These allusions include the first chapter of the Book of Job, in both situation and language, since verses are both quoted directly and paraphrased. In addition, the first three chapters of the Book of Genesis are echoed. Finally, Frost and Mordel enter into a Faustian bargain, though with better results than in the original story. The other major character is the Beta Machine, Frost's peer in the Southern Hemisphere. (Frost controls the Northern Hemisphere.) The novelette hints that though being a machine, Beta has a feminine personality. After Frost has succeeded in his millennium-long quest to become human (via recovered DNA), Beta agrees to join him in becoming human—suggesting the possibility of rebirth for the human race. The novelette has appeared in collections of Zelazny's works and in anthologies. The title is from a phrase in the poet A. E. Housman's collection A Shropshire Lad.

Social software engineering

Social software engineering (SSE) is a branch of software engineering that is concerned with the social aspects of software development and the developed software. SSE focuses on the socialness of both software engineering and developed software. On the one hand, the consideration of social factors in software engineering activities, processes and CASE tools is deemed to be useful to improve the quality of both development process and produced software. Examples include the role of situational awareness and multi-cultural factors in collaborative software development. On the other hand, the dynamicity of the social contexts in which software could operate (e.g., in a cloud environment) calls for engineering social adaptability as a runtime iterative activity. Examples include approaches which enable software to gather users' quality feedback and use it to adapt autonomously or semi-autonomously. SSE studies and builds socially-oriented tools to support collaboration and knowledge sharing in software engineering. SSE also investigates the adaptability of software to the dynamic social contexts in which it could operate and the involvement of clients and end-users in shaping software adaptation decisions at runtime. Social context includes norms, culture, roles and responsibilities, stakeholder's goals and interdependencies, end-users perception of the quality and appropriateness of each software behaviour, etc. The participants of the 1st International Workshop on Social Software Engineering and Applications (SoSEA 2008) proposed the following characterization: Community-centered: Software is produced and consumed by and/or for a community rather than focusing on individuals Collaboration/collectiveness: Exploiting the collaborative and collective capacity of human beings Companionship/relationship: Making explicit the various associations among people Human/social activities: Software is designed consciously to support human activities and to address social problems Social inclusion: Software should enable social inclusion enforcing links and trust in communities Thus, SSE can be defined as "the application of processes, methods, and tools to enable community-driven creation, management, deployment, and use of software in online environments". One of the main observations in the field of SSE is that the concepts, principles, and technologies made for social software applications are applicable to software development itself as software engineering is inherently a social activity. SSE is not limited to specific activities of software development. Accordingly, tools have been proposed supporting different parts of SSE, for instance, social system design or social requirements engineering. Consequently vertical market software, such as software development tools, engineering tools, marketing tools or software that helps users in a decision-making process can profit from social components. Such vertical social software differentiates strongly in its user-base from traditional social software such as Yammer.

AI Seoul Summit 2024

The AI Seoul Summit 2024 was an event in May 2024 co-hosted by the South Korean and British governments. The Seoul Declaration was adopted to address artificial intelligence technology and related challenges and opportunities. == Background == The AI Seoul Summit is the second such meeting following the AI Safety Summit held in the United Kingdom in November 2023. In the Bletchley Declaration, the participating countries agreed to prioritize identifying AI safety risks of shared concern, a shared concern, but at the Seoul Summit, the leaders also recognized the importance of AI. == Notable attendees == The summit was attended by the leaders of Group of Seven countries, including the United States, Canada, France, and Germany, South Korea, Singapore and Australia, representatives of the United Nations, the Organisation for Economic Co-operation and Development, and the European Union. Also in attendance were representatives of global companies such as Tesla CEO Elon Musk, Samsung Electronics Chairman Lee Jae-yong, ChatGPT maker OpenAI, Google, Microsoft, Meta, and South Korea's top portal operator Naver. == Topics == === South Korean AI safety center === "South Korea will push forward with the establishment of an AI safety research center in Korea and join a network to boost the global safety of AI." Minister of Science, Lee Jong-ho said that South Korea was planning to open an AI Safety Institute in 2024. He also expressed his intention to strengthen cooperation for the development of international standards. === Seoul Declaration for Safe, Innovative and Inclusive AI === The Seoul Declaration was adopted at the summit by leaders representing the EU, the US, the UK, Australia, Canada, Germany, France, Italy, Japan, South Korea, and Singapore. The declaration is a commitment to foster international cooperation to help develop AI governance frameworks that are interoperable between countries, partly by integrating the Hiroshima Process International Code of Conduct for Organizations Developing Advanced AI Systems. It advocates for the development of human-centric AI in collaboration with the private sector, academia, and civil society. === Seoul Ministerial Statement for advancing AI safety === At the ministerial meeting of the summit, the Seoul Ministerial Statement, a joint statement calling for the improvement of the safety, innovation, and inclusivity of AI technologies, was adopted by ministers from Australia, Canada, Chile, France, Germany, India, Indonesia, Israel, Italy, Japan, Kenya, Mexico, the Netherlands, Nigeria, New Zealand, the Philippines, South Korea, Rwanda, Saudi Arabia, Singapore, Spain, Switzerland, Turkey, Ukraine, the United Arab Emirates, the UK, and the US, as well as an EU representative. It aims to develop low-power chips as the AI industry rapidly expands and massive consumption is expected. == Global AI Summit series ==