In software engineering, more specifically in distributed computing, observability is the ability to collect data about programs' execution, modules' internal states, and the communication among components. To improve observability, software engineers use a wide range of logging and tracing techniques to gather telemetry information, and tools to analyze and use it. Observability is foundational to site reliability engineering, as it is the first step in triaging a service outage. One of the goals of observability is to minimize the amount of prior knowledge needed to debug an issue. == Etymology, terminology and definition == The term is borrowed from control theory, where the "observability" of a system measures how well its state can be determined from its outputs. Similarly, software observability measures how well a system's state can be understood from the obtained telemetry (metrics, logs, traces, profiling). The definition of observability varies by vendor: Observability is the process of making a system’s internal state more transparent. Systems are made observable by the data they produce, which in turn helps you to determine if your infrastructure or application is healthy and functioning normally. a measure of how well you can understand and explain any state your system can get into, no matter how novel or bizarre [...] without needing to ship new code software tools and practices for aggregating, correlating and analyzing a steady stream of performance data from a distributed application along with the hardware and network it runs onobservability starts by shipping all your raw data to central service before you begin analysisthe ability to measure a system’s current state based on the data it generates, such as logs, metrics, and traces Observability is tooling or a technical solution that allows teams to actively debug their system. Observability is based on exploring properties and patterns not defined in advance. proactively collecting, visualizing, and applying intelligence to all of your metrics, events, logs, and traces—so you can understand the behavior of your complex digital system The term is frequently referred to as its numeronym o11y (where 11 stands for the number of letters between the first letter and the last letter of the word). This is similar to other computer science abbreviations such as i18n and l10n and k8s. === Observability vs. monitoring === Observability and monitoring are sometimes used interchangeably. As tooling, commercial offerings and practices evolved in complexity, "monitoring" was re-branded as observability in order to differentiate new tools from the old. The terms are commonly contrasted in that systems are monitored using predefined sets of telemetry, and monitored systems may be observable. Majors et al. suggest that engineering teams that only have monitoring tools end up relying on expert foreknowledge (seniority), whereas teams that have observability tools rely on exploratory analysis (curiosity). == Telemetry types == Observability relies on three main types of telemetry data: metrics, logs and traces. Those are often referred to as "pillars of observability". === Metrics === A metric is a point in time measurement (scalar) that represents some system state. Examples of common metrics include: number of HTTP requests per second; total number of query failures; database size in bytes; time in seconds since last garbage collection. Monitoring tools are typically configured to emit alerts when certain metric values exceed set thresholds. Thresholds are set based on knowledge about normal operating conditions and experience. Metrics are typically tagged to facilitate grouping and searchability. Application developers choose what kind of metrics to instrument their software with, before it is released. As a result, when a previously unknown issue is encountered, it is impossible to add new metrics without shipping new code. Furthermore, their cardinality can quickly make the storage size of telemetry data prohibitively expensive. Since metrics are cardinality-limited, they are often used to represent aggregate values (for example: average page load time, or 5-second average of the request rate). Without external context, it is impossible to correlate between events (such as user requests) and distinct metric values. === Logs === Logs, or log lines, are generally free-form, unstructured text blobs that are intended to be human readable. Modern logging is structured to enable machine parsability. As with metrics, an application developer must instrument the application upfront and ship new code if different logging information is required. Logs typically include a timestamp and severity level. An event (such as a user request) may be fragmented across multiple log lines and interweave with logs from concurrent events. === Traces === ==== Distributed traces ==== A cloud native application is typically made up of distributed services which together fulfill a single request. A distributed trace is an interrelated series of discrete events (also called spans) that track the progression of a single user request. A trace shows the causal and temporal relationships between the services that interoperate to fulfill a request. Instrumenting an application with traces means sending span information to a tracing backend. The tracing backend correlates the received spans to generate presentable traces. To be able to follow a request as it traverses multiple services, spans are labeled with unique identifiers that enable constructing a parent-child relationship between spans. Span information is typically shared in the HTTP headers of outbound requests. === Continuous profiling === Continuous profiling is another telemetry type used to precisely determine how an application consumes resources. === Instrumentation === To be able to observe an application, telemetry about the application's behavior needs to be collected or exported. Instrumentation means generating telemetry alongside the normal operation of the application. Telemetry is then collected by an independent backend for later analysis. In fast-changing systems, instrumentation itself is often the best possible documentation, since it combines intention (what are the dimensions that an engineer named and decided to collect?) with the real-time, up-to-date information of live status in production. Instrumentation can be automatic, or custom. Automatic instrumentation offers blanket coverage and immediate value; custom instrumentation brings higher value but requires more intimate involvement with the instrumented application. Instrumentation can be native - done in-code (modifying the code of the instrumented application) - or out-of-code (e.g. sidecar, eBPF). Verifying new features in production by shipping them together with custom instrumentation is a practice called "observability-driven development". == "Pillars of observability" == Metrics, logs and traces are most commonly listed as the pillars of observability. Majors et al. suggest that the pillars of observability are high cardinality, high-dimensionality, and explorability, arguing that runbooks and dashboards have little value because "modern systems rarely fail in precisely the same way twice." == Self monitoring == Self monitoring is a practice where observability stacks monitor each other, in order to reduce the risk of inconspicuous outages. Self monitoring may be put in place in addition to high availability and redundancy to further avoid correlated failures.
Shader lamps
Shader lamps is a computer graphic technique used to change the appearance of physical objects. The still or moving objects are illuminated, using one or more video projectors, by static or animated texture or video stream. The method was invented at University of North Carolina at Chapel Hill by Ramesh Raskar, Greg Welch, Kok-lim Low and Deepak Bandyopadhyay in 1999 [1] as a follow on to Spatial Augmented Reality [2] also invented at University of North Carolina at Chapel Hill in 1998 by Ramesh Raskar, Greg Welch and Henry Fuchs. A 3D graphic rendering software is typically used to compute the deformation caused by the non perpendicular, non-planar or even complex projection surface. Complex objects (or aggregation of multiple simple objects) create self shadows that must be compensated by using several projectors. The objects are typically replaced by neutral color ones, the projection giving all its visual properties, thus the name shader lamps. The technique can be used to create a sense of invisibility, by rendering transparency. The object is illuminated not by a replacement of its own visual properties, but by the corresponding visual surface placed behind the object as seen from an arbitrary viewing point.
Data monetization
Data monetization, a form of monetization, may refer to the act of generating measurable economic benefits from available data sources (analytics). Less commonly, it may also refer to the act of monetizing data services. In the case of analytics, typically, these benefits accrue as revenue or expense savings, but may also include market share or corporate market value gains. Data monetization leverages data generated through business operations, available exogenous data or content, as well as data associated with individual actors such as that collected via electronic devices and sensors participating in the internet of things. For example, the ubiquity of the internet of things is generating location data and other data from sensors and mobile devices at an ever-increasing rate. When this data is collated against traditional databases, the value and utility of both sources of data increases, leading to tremendous potential to mine data for social good, research and discovery, and achievement of business objectives. Closely associated with data monetization are the emerging data as a service models for transactions involving data by the data item. There are three ethical and regulatory vectors involved in data monetization due to the sometimes conflicting interests of actors involved in the digital supply chain. The individual data creator who generates files and records through his own efforts or owns a device such as a sensor or a mobile phone that generates data has a claim to ownership of data. The business entity that generates data in the course of its operations, such as its transactions with financial institutions or risk factors discovered through feedback from customers also has a claim on data captured through their systems and platforms. However, the person that contributed the data may also have a legitimate claim on the data. Internet platforms and service providers, such as Google or Facebook that require a user to forgo some ownership interest in their data in exchange for use of the platform also have a legitimate claim on the data. Thus the practice of data monetization, although common since 2000, is now getting increasing attention from regulators. The European Union and the United States Congress have begun to address these issues. For instance, in the financial services industry, regulations involving data are included in the Gramm–Leach–Bliley Act and Dodd-Frank. Some individual creators of data are shifting to using personal data vaults and implementing vendor relationship management concepts as a reflection of an increasing resistance to their data being federated or aggregated and resold without compensation. Groups such as the Personal Data Ecosystem Consortium, Patient privacy rights, and others are also challenging corporate cooptation of data without compensation. Financial services companies are a relatively good example of an industry focused on generating revenue by leveraging data. Credit card issuers and retail banks use customer transaction data to improve targeting of cross-sell offers. Partners are increasingly promoting merchant based reward programs which leverage a bank’s data and provide discounts to customers at the same time. == Types of data monetization == Internal data monetization - An organization's data is used internally, resulting in economic benefit. This is commonly the case in organizations using analytics to uncover insights, resulting in improved profit, cost savings or the avoidance of risk. Internal data monetization is currently the most common form of monetization, requiring far fewer security, intellectual property, and legal precautions when compared to other types. The potential economic gains from this type of data monetization are limited by the organization's internal structure and situation. External data monetization - A person or organization makes data they possess available on a for-fee basis to external parties, or as a broker for same. This type of monetization is less common and requires various methods to distribute the data to potential buyers and consumers. However, the economic gain that results from collecting data, packaging and distributing it, can be quite large. == Steps == Identification of available data sources – this includes data currently available for monetization as well as other external data sources that may enhance the value of what’s currently available. Connect, aggregate, attribute, validate, authenticate, and exchange data - this allows data to be converted directly into actionable or revenue generating insight or services. Set terms and prices and facilitate data trading - methods for data vetting, storage, and access. For example, many global corporations have locked and siloed data storage infrastructures, which hinders efficient access to data and cooperative and real-time exchange. Perform Research and analytics – draw predictive insights from existing data as a basis for using data for to reduce risk, enhance product development or performance, or improve customer experience or business outcomes. Action and leveraging – the last phase of monetizing data includes determining alternative or improved data centric products, ideas, or services. Examples may include real-time actionable triggered notifications or enhanced channels such as web or mobile response mechanisms. == Pricing variables and factors == A fee for use of a platform to connect buyers and sellers use of a platform to configure, organize, and otherwise process data included in a data trade connecting or including a device or sensor into a data supply chain connecting and credentialing a creator of a data source and a data buyer – often through a federated identity connecting a data source to other data sources to be included in a data supply chain use of an internet service or other transmission services for uploading and downloading data – sometimes, for an individual, through a personal cloud use of encrypted keys to achieve secure data transfer use of a search algorithm specifically designed to tag data sources that contain data points of value to the data buyer linking a data creator or generator to a data collection protocol or form server actions – such as a notification – triggered by an update to a data item or data source included in a data supply chain A price or exchange or other trade value assigned by a data creator or generator to a data item or a data source offered by a data buyer to a data creator assigned by a data buyer for a data item or a data source formatted according to criteria set by a data buyer An incremental fee assigned by a data buyer for a data item or a data set scaled to the reputation of the data creator == Benefits == Improved decision-making that leads to real time crowd sourced research, improved profits, decreased costs, reduced risk and improved compliance More impactful decisions (e.g., make real-time decisions) More timely (lower latency) decisions (e.g., a vendor making purchase recommendations while the customer is still on the phone or in the store, a customer connecting with multiple vendors to discover the best price, triggered notifications when thresholds are reached for data values) More granular decisions (e.g., localized pricing decisions at an individual or device or sensor level versus larger aggregates). Targeted Marketing (e.g., Vendors with access to big data can make targeted advertisements to specific customers within a set data pool decreasing costs for the advertiser and reaching most interested customers) == Frameworks == There are a wide variety of industries, firms and business models related to data monetization. The following frameworks have been offered to help understand the types of business models that are used: Roger Ehrenberg of IA Ventures, a venture capital firm that invests in this sector, has defined three basic types of data product firms: Contributory databases. The magic of these businesses is that a customer provides their own data in exchange for receiving a more robust set of aggregated data back that provides insight into the broader marketplace, or provides a vehicle for expressing a view. Give a little, get a lot back in return – a pretty compelling value proposition, and one that frequently results in a payment from the data contributor in exchange for receiving enriched, aggregated data. Once these contributory databases are developed and customers become reliant on their insights, they become extremely valuable and persistent data assets. Data processing platforms. These businesses create barriers through a combination of complex data architectures, proprietary algorithms, and rich analytics to help customers consume data in whatever form they please. Often these businesses have special relationships with key data providers, that when combined with other data and processed as a whole create valuable differentiation and competitive barriers. Bloomberg is an example of a powerful
Batch cryptography
Batch cryptography is a field of cryptology focused on the design of cryptographic protocols that perform operations—such as encryption, decryption, key exchange, and authentication—on multiple inputs simultaneously, rather than processing each input individually. Batching cryptographic operations can significantly reduce the marginal cost of handling individual inputs—a principle that was first introduced by Amos Fiat in 1989.
ESign (India)
Aadhaar eSign is an online electronic signature service in India to facilitate an Aadhaar holder to digitally sign a document. The signature service is facilitated by authenticating the Aadhaar holder via the Aadhaar-based e-KYC (electronic Know Your Customer) service. To eSign a document, one has to have an Aadhaar card and a mobile number registered with Aadhaar. With these two things, an Indian citizen can sign a document remotely without being physically present. == Procedure == The notification issued by Government of India in this regard stipulates the following procedure for the e-authentication using Aadhaar e-KYC services. Authentication of an electronic record by e-authentication technique, which shall be done by the applicable use of e-authentication, hash function, and asymmetric cryptosystem techniques, leading to issuance of digital signature certificate by Certifying Authority, a trusted third party service by subscriber's key pair generation, storing of the key pairs on hardware security module and creation of digital signature provided that the trusted third party shall be offered by the certifying authority (the trusted third party shall send application form and certificate signing request to the Certifying Authority for issuing a digital signature certificate to the subscriber), issuance of digital signature certificate by Certifying Authority shall be based on e-authentication, particulars given in the prescribed format, digitally signed verified information from Aadhaar e-KYC services and electronic consent of digital signature certificate applicant, the manner and requirements for e-authentication shall be as issued by the Controller from time to time, the security procedure for creating the subscriber's key pair shall be in accordance with the e-authentication guidelines issued by the Controller, the standards referred to in rule 6 of the Information Technology (Certifying Authorities) Rules, 2000 shall be complied with, in so far as they relate to the certification function of public key of Digital Signature Certificate applicant, and the manner in which information is authenticated by means of digital signature shall comply with the standards specified in rule 6 of the Information Technology (Certifying Authorities) Rules, 2000 in so far as they relate to the creation, storage and transmission of Digital Signature. == eSign Service Providers == Organisations and individuals seeking to obtain the eSigning Service can utilize the services of various service providers. There are empanelled service providers with whom organisations can register as an Application Service Prover after submitting the requisite documents, getting UAT access, building the application around the service and going through an IT Audit by an CERT-IN empanelled auditor. However, the process of registering as an Application Service Provider is cumbersome, and requires huge investments of time, money and resources in complying with the regulations and building a suitable application. Most organisations prefer using services of plug-n-play gateway providers who take the responsibility of complying with the regulations, hence simplifying the process for the market.
Wetware (brain)
Wetware is a term drawn from the computer-related idea of hardware or software, but applied to biological life forms. == Usage == The prefix "wet" is a reference to the water found in living creatures. Wetware is used to describe the elements equivalent to hardware and software found in a person, especially the central nervous system (CNS) and the human mind. The term wetware finds use in works of fiction, in scholarly publications and in popularizations. The "hardware" component of wetware concerns the bioelectric and biochemical properties of the CNS, specifically the brain. If the sequence of impulses traveling across the various neurons are thought of symbolically as software, then the physical neurons would be the hardware. The amalgamated interaction of this software and hardware is manifested through continuously changing physical connections, and chemical and electrical influences that spread across the body. The process by which the mind and brain interact to produce the collection of experiences that we define as self-awareness is in question. == History == Although the exact definition has shifted over time, the term Wetware and its fundamental reference to "the physical mind" has been around at least since the mid-1950s. Mostly used in relatively obscure articles and papers, it was not until the heyday of cyberpunk, however, that the term found broad adoption. Among the first uses of the term in popular culture was the Bruce Sterling novel Schismatrix (1985) and the Michael Swanwick novel Vacuum Flowers (1987). Rudy Rucker references the term in a number of books, including one entitled Wetware (1988): ... all sparks and tastes and tangles, all its stimulus/response patterns – the whole bio-cybernetic software of mind. Rucker did not use the word to simply mean a brain, nor in the human-resources sense of employees. He used wetware to stand for the data found in any biological system, analogous perhaps to the firmware that is found in a ROM chip. In Rucker's sense, a seed, a plant graft, an embryo, or a biological virus are all wetware. DNA, the immune system, and the evolved neural architecture of the brain are further examples of wetware in this sense. Rucker describes his conception in a 1992 compendium The Mondo 2000 User's Guide to the New Edge, which he quotes in a 2007 blog entry. Early cyber-guru Arthur Kroker used the term in his blog. With the term getting traction in trendsetting publications, it became a buzzword in the early 1990s. In 1991, Dutch media theorist Geert Lovink organized the Wetware Convention in Amsterdam, which was supposed to be an antidote to the "out-of-body" experiments conducted in high-tech laboratories, such as experiments in virtual reality. Timothy Leary, in an appendix to Info-Psychology originally written in 1975–76 and published in 1989, used the term wetware, writing that "psychedelic neuro-transmitters were the hot new technology for booting-up the 'wetware' of the brain". Another common reference is: "Wetware has 7 plus or minus 2 temporary registers." The numerical allusion is to a classic 1957 article by George A. Miller, The magical number 7 plus or minus two: some limits in our capacity for processing information, which later gave way to Miller's law.
Private message
In computer networking, a private message (PM), or direct message (DM), refers to a private communication, often text-based, sent or received by a user of a private communication channel on any given platform. Unlike public posts, PMs are only viewable by the participants. Long a function present on IRCs and Internet forums, private channels for PMs have also been prevalent features on instant messaging (IM) and on social media networks. It may be either synchronous (e.g. on an IM) or asynchronous (e.g. on an Internet forum). The term private message (PM) originated as a feature on internet forums, while the term direct message (DM) originated as a feature on Twitter. Due to the popularity of the latter service, DM has since been appropriated by other platforms, such as Instagram, and is often genericized in popular usage. == Overview == There are two main types of private messages, and one obscure type: One type includes those found on IRCs and Internet forums, as well as on social media services like Twitter, Facebook, and Instagram, where the focus is public posting, PMs allow users to communicate privately without leaving the platform. The second type are those relayed through instant messaging platforms such as WhatsApp and Snapchat, where users join the networks primarily to exchange PMs. A third type, peer-to-peer messaging, occurs when users create and own the infrastructure used to transmit and store the messages; while features vary depending on application, they give the user full control over the data they transmit. An example of software that enables this kind of messaging is Classified-ads. Besides serving as a tool to connect privately with friends and family, PMs have gained momentum in the workplace. Working professionals use PMs to reach coworkers in other spaces and increase efficiency during meetings. Although useful, using PMs in the workplace may blur the boundary between work and private lives. Some common forms of private messaging today include Facebook messaging (sometimes referred to as "inboxing"), Twitter direct messaging, and Instagram direct messaging. These forms of private messaging provide a private space on a usually public site. For instance, most activity on Twitter is public, but Twitter DMs provide a private space for communication between two users. This differs from mediums like email, texting, and Snapchat, where most or all activity is always private. Modern forms of private messaging may include multimedia messages, such as pictures or videos. == History == Email was first developed to send messages between different computers on ARPANET in 1971. Access to ARPANET was primarily limited to universities and other research institutions. Starting in 1983 or 1984, FidoNet allowed home computer users to send and receive email via bulletin board systems. Information services such as CompuServe, America Online, and Prodigy also helped to popularizes online messaging. The advent of the public World Wide Web in 1993 increased access to email via internet service providers, and later via webmail. Instant messaging systems became popular in the mid 1990s, as Internet access improved and personal computers became more common. The introduction of Skype in 2003 popularized Internet-based voice and video messaging. Direct messaging is now a feature of all major social networking services. == Privacy concerns == In January 2014, Matthew Campbell and Michael Hurley filed a class-action lawsuit against Facebook for breaching the Electronic Communications Privacy Act. They alleged that private messages which contained URLs were being read and used to generate profit, through data mining and user profiling, and that it was misleading for Facebook to refer to the functionality as "private" with the implication that the communication was "free from surveillance". In 2012, some Facebook users misinterpreted a redesign of the Facebook wall as publicly sharing private messages from 2008–2009. These were found to be public wall posts from those years, made at a time when it was not possible to like or comment on a wall post, making the notes look like private messages.