AI App UI Design

AI App UI Design — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.

• 

  Digital video effect

  Digital video effects (DVEs) are visual effects that provide comprehensive live video image manipulation, in the same form as optical printer effects in film. DVEs differ from standard video switcher effects (often referred to as analog effects) such as wipes or dissolves, in that they deal primarily with resizing, distortion or movement of the image. Modern video switchers often contain internal DVE functionality. Modern DVE devices are incorporated in high-end broadcast video switchers. Early examples of DVE devices found in the broadcast post-production industry include the Ampex Digital Optics (ADO), Quantel DPE-5000, Vital Squeezoom, NEC E-Flex and the Abekas A5x series of DVEs. By 1988, Grass Valley Group caught up with the competition with their Kaleidoscope, which integrated ADO-type effects with their widely used line of broadcast switching gear. DVEs are used by the broadcast television industry in live television production environments like television studios and outside broadcasts. They are commonly used in video post-production.

  Read more →
• 

  Social media mining

  Social media mining is the process of obtaining data from user-generated content on social media in order to extract actionable patterns, form conclusions about users, and act upon the information. Mining supports targeting advertising to users or academic research. The term is an analogy to the process of mining for minerals. Mining companies sift through raw ore to find the valuable minerals; likewise, social media mining sifts through social media data in order to discern patterns and trends about matters such as social media usage, online behaviour, content sharing, connections between individuals, buying behaviour. These patterns and trends are of interest to companies, governments and not-for-profit organizations, as such organizations can use the analyses for tasks such as design strategies, introduce programs, products, processes or services. Social media mining uses concepts from computer science, data mining, machine learning, and statistics. Mining is based on social network analysis, network science, sociology, ethnography, optimization and mathematics. It attempts to formally represent, measure and model patterns from social media data. In the 2010s, major corporations, governments and not-for-profit organizations began mining to learn about customers, clients and others. Platforms such as Google, Facebook (partnered with Datalogix and BlueKai) conduct mining to target users with advertising. Scientists and machine learning researchers extract insights and design product features. Users may not understand how platforms use their data. Users tend to click through Terms of Use agreements without reading them, leading to ethical questions about whether platforms adequately protect users' privacy. During the 2016 United States presidential election, Facebook allowed Cambridge Analytica, a political consulting firm linked to the Trump campaign, to analyze the data of an estimated 87 million Facebook users to profile voters, creating controversy when this was revealed. == Background == As defined by Kaplan and Haenlein, social media is the "group of internet-based applications that build on the ideological and technological foundations of Web 2.0, and that allow the creation and exchange of user-generated content." There are many categories of social media including, but not limited to, social networking (Facebook or LinkedIn), microblogging (Twitter), photo sharing (Flickr, Instagram, Photobucket, or Picasa), news aggregation (Google Reader, StumbleUpon, or Feedburner), video sharing (YouTube, MetaCafe), livecasting (Ustream or Twitch), virtual worlds (Kaneva), social gaming (World of Warcraft), social search (Google, Bing, or Ask.com), and instant messaging (Google Talk, Skype, or Yahoo! messenger). The first social media website was introduced by GeoCities in 1994. It enabled users to create their own homepages without having a sophisticated knowledge of HTML coding. The first social networking site, SixDegrees.com, was introduced in 1997. Since then, many other social media sites have been introduced, each providing service to millions of people. These individuals form a virtual world in which individuals (social atoms), entities (content, sites, etc.) and interactions (between individuals, between entities, between individuals and entities) coexist. Social norms and human behavior govern this virtual world. By understanding these social norms and models of human behavior and combining them with the observations and measurements of this virtual world, one can systematically analyze and mine social media. Social media mining is the process of representing, analyzing, and extracting meaningful patterns from data in social media, resulting from social interactions. It is an interdisciplinary field encompassing techniques from computer science, data mining, machine learning, social network analysis, network science, sociology, ethnography, statistics, optimization, and mathematics. Social media mining faces grand challenges such as the big data paradox, obtaining sufficient samples, the noise removal fallacy, and evaluation dilemma. Social media mining represents the virtual world of social media in a computable way, measures it, and designs models that can help us understand its interactions. In addition, social media mining provides necessary tools to mine this world for interesting patterns, analyze information diffusion, study influence and homophily, provide effective recommendations, and analyze novel social behavior in social media. == Uses == Social media mining is used across several industries including business development, social science research, health services, and educational purposes. Once the data received goes through social media analytics, it can then be applied to these various fields. Often, companies use the patterns of connectivity that pervade social networks, such as assortativity—the social similarity between users that are induced by influence, homophily, and reciprocity and transitivity. These forces are then measured via statistical analysis of the nodes and connections between these nodes. Social analytics also uses sentiment analysis, because social media users often relay positive or negative sentiment in their posts. This provides important social information about users' emotions on specific topics. These three patterns have several uses beyond pure analysis. For example, influence can be used to determine the most influential user in a particular network. Companies would be interested in this information in order to decide who they may hire for influencer marketing. These influencers are determined by recognition, activity generation, and novelty—three requirements that can be measured through the data mined from these sites. Analysts also value measures of homophily: the tendency of two similar individuals to become friends. Users have begun to rely on information of other users' opinions in order to understand diverse subject matter. These analyses can also help create recommendations for individuals in a tailored capacity. By measuring influence and homophily, online and offline companies are able to suggest specific products for individuals consumers, and groups of consumers. Social media networks can use this information themselves to suggest to their users possible friends to add, pages to follow, and accounts to interact with. == Perception == Modern social media mining is a controversial practice that has led to exponential gains in user growth for tech giants such as Facebook, Inc., Twitter, and Google. Companies such as these, considered "Big Tech" are companies that build algorithms that take advantage of user input to understand their preferences, and keep them on the platform as much as possible. These inputs, that can be as simple as time spent on a given screen, provide the data being mined, and lead to companies profiting heavily from using that data to capitalize on extremely accurate predictions about user behavior. The growth of platforms accelerated rapidly once these strategies were put in place; Most of the largest platforms now average over 1 billion active users per month as of 2021. It has been claimed by a multitude of anti-algorithm personalities, like Tristan Harris or Chamath Palihapitiya, that certain companies (specifically Facebook) valued growth above all else, and ignored potential negative impacts from these growth engineering tactics. At the same time, users have now created their own data arbitrages with the help of their own data, through content monetization and becoming influencers. Users typically have access to a varied set of analytics specific to people that interact with them on social media, and can use these as building blocks for their own targeting and growth strategies through ads and posts that cater to their audiences. Influencers also commonly promote products and services for established brands, creating one of the largest digital industries: Influencer marketing. Instagram, Facebook, Twitter, YouTube, Google, and others have long given access to platform analytics, and allowed third parties to access that information as well, at times unbeknownst to even the user whose data is being viewed/bought. == Research == === Research areas === Social media event detection – Social networks enable users to freely communicate with each other and share their recent news, ongoing activities or views about different topics. As a result, they can be seen as a potentially viable source of information to understand the current emerging topics/events. Public health monitoring and surveillance - Using large-scale analysis of social media to study large cohorts of patients and the general public, e.g. to obtain early warning signals of drug-drug interactions and adverse drug reactions, or understand human reproduction and sexual interest. Community structure (Community Detection/Evolution/Evaluation) – Identifying communities on social networks, how t

  Read more →
• 

  Information leakage

  Information leakage happens whenever a system that is designed to be closed to an eavesdropper reveals some information to unauthorized parties nonetheless. In other words: Information leakage occurs when secret information correlates with, or can be correlated with, observable information. For example, when designing an encrypted instant messaging network, a network engineer without the capacity to crack encryption codes could see when messages are transmitted, even if he could not read them. == Risk vectors == A modern example of information leakage is the leakage of secret information via data compression, by using variations in data compression ratio to reveal correlations between known (or deliberately injected) plaintext and secret data combined in a single compressed stream. Another example is the key leakage that can occur when using some public-key systems when cryptographic nonce values used in signing operations are insufficiently random. Bad randomness cannot protect proper functioning of a cryptographic system, even in a benign circumstance, it can easily produce crackable keys that cause key leakage. Information leakage can sometimes be deliberate: for example, an algorithmic converter may be shipped that intentionally leaks small amounts of information, in order to provide its creator with the ability to intercept the users' messages, while still allowing the user to maintain an illusion that the system is secure. This sort of deliberate leakage is sometimes known as a subliminal channel. Generally, only very advanced systems employ defenses against information leakage. Following are the commonly implemented countermeasures : Use steganography to hide the fact that a message is transmitted at all. Use chaffing to make it unclear to whom messages are transmitted (but this does not hide from others the fact that messages are transmitted). For busy re-transmitting proxies, such as a Mixmaster node: randomly delay and shuffle the order of outbound packets - this will assist in disguising a given message's path, especially if there are multiple, popular forwarding nodes, such as are employed with Mixmaster mail forwarding. When a data value is no longer going to be used, erase it from the memory.

  Read more →
• 

  Social employee

  A social employee is a worker operating within a social business model. Following an organization's social computing guidelines, social employees use social media tools both for internal workflow and collaboration purposes and for external engagement with customers, prospects and stakeholders through a combination of social media marketing, content marketing, social marketing, and social selling. Social employee programs are considered to be as much about culture and engagement as they are about business processes and best practices. In addition to increased leads and sales, social employee best practices are said to improve business outcomes important to social media marketing, such as increased connections and web traffic, improved brand identification and "chatter", and better customer advocacy. == Overview == The term "social employee" was first introduced to describe those exhibiting the emerging characteristics of workers operating under a social business model. The term is often used interchangeably with similar designations like "employee advocate" or "social employee advocate". Crucial to the perceived value of the social employee is the concept of the digital footprint. While organizations are able to generate large bases of followers through social media, research shows that brand marketing and engagement efforts through these networks are not as effective as those of individual employees. In fact, some research indicates that employee experts are more trusted than any other member of an organization. Because of this, social employee programs are designed to train, empower, and support employee engagement efforts in the hopes of authentically engaging larger communities, increasing the frequency of shares, reviews, and other forms of "earned media" and expanding the brand's presence on the web. == The personal or employee brand == A foundational concept of the social employee is the idea of the personal or employee brand. This concept first gained popular attention in a 1997 FastCompany article by business leader Tom Peters titled "The Brand Called You". In the article, Peters argued that the premium placed on branding impacted workers' lives to such an extent that creating and cultivating a distinct personal brand had become a professional necessity. According to Peters, doing so built trust, loyalty, visibility, influence, and employability. With increased adoption of social media tools by both businesses and consumers in the early 21st century, many business leaders became increasingly concerned with social engagement, both internally among employees and externally with customers and other stakeholders. While many in the business community acknowledged the potential social tools had for improved collaboration, productivity, and brand messaging, the concern that employees would misrepresent their brand, disclose proprietary information, or otherwise damage their company's reputation or ability to conduct business persisted. As a result, many began to advocate for employee branding as a solution to this problem. This helped give new meaning to the concept of brand ambassadorship, positioning everyday employees in public, and potentially high-profile, engagement roles. == Characteristics == === Engaged === Because social employee advocacy is dependent on the perceived authenticity of the employee, engagement is highly valued in social organizations. Further, data show the measurable impact of employee engagement on organizational productivity and profitability: Happy employees were found to be 12 percent more productive. In one study, engaged employees were found to be 38 percent more likely to produce at above-average rates. In another, organizations with engaged employees had a 19 percent higher than average shareholder return, while organizations with disengaged employees experienced shareholder return that was 44 percent below average. Engaged companies were found to outperform disengaged companies by up to 202 percent. Companies with strong focus on culture were found to have an average 13.9 percent turnover rate, while those with a low focus experience were found to have a 48.4 percent turnover rate. === Flexible job environment and work–life balance === The number of professionals working mobile or remote has risen considerably since 2010. While estimates vary, one study found that number of organizations with mobile or remote employees is expected to rise from 24 percent in 2012 to 89 percent by 2020. Other research has estimated that by 2020, 105.4 million professionals will work remotely in America, about 72.3 percent of the total workforce. This change has been linked to a rise in social technologies, including biometrics, wearables, near-field communications, and augmented reality. Social employees have also put a greater emphasis on work–life balance, with many believing that advances in technology can directly support efforts in this area. Purported benefits of this shift include a more flexible workforce, reduced business costs, and greater organizational leverage in attracting and retaining top talent. === Buys into the brand's story === In 2009, thought leader Simon Sinek presented a speech called "How Great Leaders Inspire Action" at a TEDxPugetSound event. Sinek's central argument in this speech was, "People don't buy what you do. They buy why you do it." This concept—that the story behind a business or product offering is a more compelling sales tool than the product itself—is frequently cited in social media marketing as a way to build authentic connections with stakeholders. However, others have argued that for employees to share a brand's story authentically, they must be engaged in that story themselves, and as a result, many companies have made storytelling part of their culture programs. === Collaborative === An implicit tenet in social business is that social technologies aren't a barrier to productivity, but rather a path to increased connectivity. The shift in enterprise software systems like IBM Connections to incorporate social communication models, such as mentions, wikis, and newsfeeds, reflects the changing communication dynamics within business. With an increase in diversity and sophistication in collaborative software platforms, social organizations have sought to find new creative ways to utilize these tools and secure employee buy-in around them. Crowdsourcing has also become popular in social businesses. Examples include AT&T's program The Innovation Pipeline (TIP), begun in 2009, which has generated over 28,000 ideas that have led to over 75 projects with funding exceeding $44 million. IBM has also put considerable resources into such processes, producing its social computing guidelines through employee crowdsourcing, as well as its Connections platform through the Technology Adoption Program (TAP), a more formalized crowdsourcing initiative. Another popular form of internal collaboration is the hack day, or hackathon. Organizations such as Netflix, Facebook, and IBM use hack days to pull employees out of their day-to-day work environments and encourage them to collaborate in nontraditional ways in an attempt to drive disruptive innovation. Social employees are often encouraged to seek external collaboration opportunities with customers and prospects. For example, Procter & Gamble introduced the Live Well Collaborative to connect with external stakeholders and develop products and services for the 50+ demographic. === Social listener === A social listener is someone who engages in social listening, or social media monitoring, for professional means. Social employees can use social media monitoring for a variety of reasons, including professional development, industry news and trends, and gauging market sentiment. Some have argued that social listening is one of the most important components of social business, as it enables organizations to collect rich market data, make more informed strategic decisions, and respond to customer needs more authentically. === Customer-centric === Advocates of customer-centricity in social business argue that social media has changed the dynamic from one-way brand messaging to shared interactions between brand and customer. Brand and customer engagement is seen as a means of creating more lasting connections with customers and prospects and empowering them to become brand promoters. Customer-centric interactions are seen to have distinct value to brands, as research shows that prospects are far more likely to trust brand-related messaging from a friend or family member than they are from a brand. As a means of building social employees, some social advocates have also called for a broader definition of customer to include the employees themselves. In the book The Pursuit of Social Business Excellence, authors Vala Afshar and Brad Martin made the following argument: A social business operates with the guiding principle that each employee's responsi

  Read more →
• 

  Audio-visual speech recognition

  Audio visual speech recognition (AVSR) is a technique that uses image processing capabilities in lip reading to aid speech recognition systems in recognizing indeterministic phones or giving preponderance among near probability decisions. Each system of lip reading and speech recognition works separately, then their results are mixed at the stage of feature fusion. As the name suggests, it has two parts. First one is the audio part and second one is the visual part. In audio part we use features like log mel spectrogram, mfcc etc. from the raw audio samples and we build a model to get feature vector out of it . For visual part generally we use some variant of convolutional neural network to compress the image to a feature vector after that we concatenate these two vectors (audio and visual ) and try to predict the target object.

  Read more →
• 

  Application delivery network

  An application delivery network (ADN) is a suite of technologies that, when deployed together, provide availability, security, visibility, and acceleration for Internet applications such as websites. ADN components provide supporting functionality that enables website content to be delivered to visitors and other users of that website, in a fast, secure, and reliable way. Gartner defines application delivery networking as the combination of WAN optimization controllers (WOCs) and application delivery controllers (ADCs). At the data center end of an ADN is the ADC, an advanced traffic management device that is often also referred to as a web switch, content switch, or multilayer switch, the purpose of which is to distribute traffic among a number of servers or geographically dislocated sites based on application specific criteria. In the branch office portion of an ADN is the WAN optimization controller, which works to reduce the number of bits that flow over the network using caching and compression, and shapes TCP traffic using prioritization and other optimization techniques. Some WOC components are installed on PCs or mobile clients, and there is typically a portion of the WOC installed in the data center. Application delivery networks are also offered by some CDN vendors. The ADC, one component of an ADN, evolved from layer 4-7 switches in the late 1990s when it became apparent that traditional load balancing techniques were not robust enough to handle the increasingly complex mix of application traffic being delivered over a wider variety of network connectivity options. == Application delivery techniques == The Internet was designed according to the end-to-end principle. This principle keeps the core network relatively simple and moves the intelligence as much as possible to the network end-points: the hosts and clients. An Application Delivery Network (ADN) enhances the delivery of applications across the Internet by employing a number of optimization techniques. Many of these techniques are based on established best-practices employed to efficiently route traffic at the network layer including redundancy and load balancing In theory, an Application Delivery Network (ADN) is closely related to a content delivery network. The difference between the two delivery networks lies in the intelligence of the ADN to understand and optimize applications, usually referred to as application fluency. Application Fluent Network (AFN) is based on the concept of Application Fluency to refer to WAN optimization techniques applied at Layer Four to Layer Seven of the OSI model for networks. Application Fluency implies that the network is fluent or intelligent in understanding and being able to optimize delivery of each application. Application Fluent Network is an addition of SDN capabilities. The acronym 'AFN' is used by Alcatel-Lucent Enterprise to refer to an Application Fluent Network. Application delivery uses one or more layer 4–7 switches, also known as a web switch, content switch, or multilayer switch to intelligently distribute traffic to a pool, also known as a cluster or farm, of servers. The application delivery controller (ADC) is assigned a single virtual IP address (VIP) that represents the pool of servers. Traffic arriving at the ADC is then directed to one of the servers in the pool (cluster, farm) based on a number of factors including application specific data values, application transport protocol, availability of servers, current performance metrics, and client-specific parameters. An ADN provides the advantages of load distribution, increase in capacity of servers, improved scalability, security, and increased reliability through application specific health checks. Increasingly the ADN comprises a redundant pair of ADC on which is integrated a number of different feature sets designed to provide security, availability, reliability, and acceleration functions. In some cases these devices are still separate entities, deployed together as a network of devices through which application traffic is delivered, each providing specific functionality that enhances the delivery of the application. == ADN optimization techniques == === TCP multiplexing === TCP Multiplexing is loosely based on established connection pooling techniques utilized by application server platforms to optimize the execution of database queries from within applications. An ADC establishes a number of connections to the servers in its pool and keeps the connections open. When a request is received by the ADC from the client, the request is evaluated and then directed to a server over an existing connection. This has the effect of reducing the overhead imposed by establishing and tearing down the TCP connection with the server, improving the responsiveness of the application. Some ADN implementations take this technique one step further and also multiplex HTTP and application requests. This has the benefit of executing requests in parallel, which enhances the performance of the application. === TCP optimization === There are a number of Request for Comments (RFCs) which describe mechanisms for improving the performance of TCP. Many ADN implement these RFCs in order to provide enhanced delivery of applications through more efficient use of TCP. The RFCs most commonly implemented are: Delayed Acknowledgements Nagle Algorithm Selective Acknowledgements Explicit Congestion Notification ECN Limited and Fast Retransmits Adaptive Initial Congestion Windows === Data compression and caching === ADNs also provide optimization of application data through caching and compression techniques. There are two types of compression used by ADNs today: industry standard HTTP compression and proprietary data reduction algorithms. It is important to note that the cost in CPU cycles to compress data when traversing a LAN can result in a negative performance impact and therefore best practices are to only utilize compression when delivering applications via a WAN or particularly congested high-speed data link. HTTP compression is asymmetric and transparent to the client. Support for HTTP compression is built into web servers and web browsers. All commercial ADN products currently support HTTP compression. A second compression technique is achieved through data reduction algorithms. Because these algorithms are proprietary and modify the application traffic, they are symmetric and require a device to reassemble the application traffic before the client can receive it. A separate class of devices known as WAN Optimization Controllers (WOC) provide this functionality, but the technology has been slowly added to the ADN portfolio over the past few years as this class of device continues to become more application aware, providing additional features for specific applications such as CIFS and SMB. == ADN reliability and availability techniques == === Advanced health checking === Advanced health checking is the ability of an ADN to determine not only the state of the server on which an application is hosted, but the status of the application it is delivering. Advanced health checking techniques allow the ADC to intelligently determine whether or not the content being returned by the server is correct and should be delivered to the client. This feature enables other reliability features in the ADN, such as resending a request to a different server if the content returned by the original server is found to be erroneous. === Load balancing algorithms === The load balancing algorithms found in today's ADN are far more advanced than the simplistic round-robin and least connections algorithms used in the early 1990s. These algorithms were originally loosely based on operating systems' scheduling algorithms, but have since evolved to factor in conditions peculiar to networking and application environments. It is more accurate to describe today's "load balancing" algorithms as application routing algorithms, as most ADN employ application awareness to determine whether an application is available to respond to a request. This includes the ability of the ADN to determine not only whether the application is available, but whether or not the application can respond to the request within specified parameters, often referred to as a service level agreement. Typical industry standard load balancing algorithms available today include: Round Robin Least Connections Fastest Response Time Weighted Round Robin Weighted Least Connections Custom values assigned to individual servers in a pool based on SNMP or other communication mechanism === Fault tolerance === The ADN provides fault tolerance at the server level, within pools or farms. This is accomplished by designating specific servers as a 'backup' that is activated automatically by the ADN in the event that the primary server(s) in the pool fail. The ADN also ensures application availability and reliability through its ability to seamlessly "failover"

  Read more →
• 

  Torus interconnect

  A torus interconnect is a switch-less network topology for connecting processing nodes in a parallel computer system. == Introduction == In geometry, a torus is created by revolving a circle about an axis coplanar to the circle. While this is a general definition in geometry, the topological properties of this type of shape describes the network topology in its essence. === Geometry illustration === In the representations below, the first is a one dimension torus, a simple circle. The second is a two dimension torus, in the shape of a 'doughnut'. The animation illustrates how a two dimension torus is generated from a rectangle by connecting its two pairs of opposite edges. At one dimension, a torus topology is equivalent to a ring interconnect network, in the shape of a circle. At two dimensions, it becomes equivalent to a two dimension mesh, but with extra connection at the edge nodes. === Torus network topology === A torus interconnect is a switch-less topology that can be seen as a mesh interconnect with nodes arranged in a rectilinear array of N = 2, 3, or more dimensions, with processors connected to their nearest neighbors, and corresponding processors on opposite edges of the array connected.[1] In this lattice, each node has 2N connections. This topology is named for the lattice formed in this way, which is topologically homogeneous to an N-dimensional torus. == Visualization == The first 3 dimensions of torus network topology are easier to visualize and are described below: 1D Torus: one dimension, n nodes are connected in closed loop with each node connected to its two nearest neighbors. Communication can take place in two directions, +x and −x. A 1D Torus is the same as ring interconnection. 2D Torus: two dimensions with degree of four, the nodes are imagined laid out in a two-dimensional rectangular lattice of n rows and n columns, with each node connected to its four nearest neighbors, and corresponding nodes on opposite edges connected. Communication can take place in four directions, +x, −x, +y, and −y. The total nodes of a 2D Torus is n2. 3D Torus: three dimensions, the nodes are imagined in a three-dimensional lattice in the shape of a rectangular prism, with each node connected with its six neighbors, with corresponding nodes on opposing faces of the array connected. Each edge consists of n nodes. communication can take place in six directions, +x, −x, +y, −y, +z, −z. Each edge of a 3D Torus consist of n nodes. The total nodes of 3D Torus is n3. ND Torus: N dimensions, each node of an N dimension torus has 2N neighbors, Communication can take place in 2N directions. Each edge consists of n nodes. Total nodes of this torus is nN. The main motivation of having higher dimension of torus is to achieve higher bandwidth, lower latency, and higher scalability. Higher-dimensional arrays are difficult to visualize. The above ruleset shows that each higher dimension adds another pair of nearest neighbor connections to each node. == Performance == A number of supercomputers on the TOP500 list use three-dimensional torus networks, e.g. IBM's Blue Gene/L and Blue Gene/P, and the Cray XT3. IBM's Blue Gene/Q uses a five-dimensional torus network. Fujitsu's K computer and the PRIMEHPC FX10 use a proprietary three-dimensional torus 3D mesh interconnect called Tofu. === 3D Torus performance simulation === Sandeep Palur and Dr. Ioan Raicu from Illinois Institute of Technology conducted experiments to simulate 3D torus performance. Their experiments ran on a computer with 250GB RAM, 48 cores and x86_64 architecture. The simulator they used was ROSS (Rensselaer’s Optimistic Simulation System). They mainly focused on three aspects: Varying network size Varying number of servers Varying message size They concluded that throughput decreases with the increase of servers and network size. Otherwise, throughput increases with the increase of message size. === 6D Torus product performance === Fujitsu Limited developed a 6D torus computer model called "Tofu". In their model, a 6D torus can achieve 100 GB/s off-chip bandwidth, 12 times higher scalability than a 3D torus, and high fault tolerance. The model is used in the K computer and Fugaku. === Cost === While long wrap-around links may be the easiest way to visualize the connection topology, in practice, restrictions on cable lengths often make long wrap-around links impractical. Instead, directly connected nodes—including nodes that the above visualization places on opposite edges of a grid, connected by a long wrap-around link—are physically placed nearly adjacent to each other in a folded torus network. Every link in the folded torus network is very short—almost as short as the nearest-neighbor links in a simple grid interconnect—and therefore low-latency.

  Read more →
• 

  Forward anonymity

  Forward anonymity is a property of a cryptographic system which prevents an attacker who has recorded past encrypted communications from discovering its contents and participants in the future. This property is analogous to forward secrecy. An example of a system which uses forward anonymity is a public key cryptography system, where the public key is well-known and used to encrypt a message, and an unknown private key is used to decrypt it. In this system, one of the keys is always said to be compromised, but messages and their participants are still unknown by anyone without the corresponding private key. In contrast, an example of a system which satisfies the perfect forward secrecy property is one in which a compromise of one key by an attacker (and consequent decryption of messages encrypted with that key) does not undermine the security of previously used keys. Forward secrecy does not refer to protecting the content of the message, but rather to the protection of keys used to decrypt messages. == History == Originally introduced by Whitfield Diffie, Paul van Oorschot, and Michael James Wiener to describe a property of STS (station-to-station protocol) involving a long term secret, either a private key or a shared password. == Public Key Cryptography == Public Key Cryptography is a common form of a forward anonymous system. It is used to pass encrypted messages, preventing any information about the message from being discovered if the message is intercepted by an attacker. It uses two keys, a public key and a private key. The public key is published, and is used by anyone to encrypt a plaintext message. The Private key is not well known, and is used to decrypt cyphertext. Public key cryptography is known as an asymmetric decryption algorithm because of different keys being used to perform opposing functions. Public key cryptography is popular because, while it is computationally easy to create a pair of keys, it is extremely difficult to determine the private key knowing only the public key. Therefore, the public key being well known does not allow messages which are intercepted to be decrypted. This is a forward anonymous system because one compromised key (the public key) does not compromise the anonymity of the system. == Web of Trust == A variation of the public key cryptography system is a Web of trust, where each user has both a public and private key. Messages sent are encrypted using the intended recipient's public key, and only this recipient's private key will decrypt the message. They are also signed with the senders private key. This creates added security where it becomes more difficult for an attacker to pretend to be a user, as the lack of a private key signature indicates a non-trusted user. == Limitations == A forward anonymous system does not necessarily mean a wholly secure system. A successful cryptanalysis of a message or sequence of messages can still decode the information without the use of a private key or long term secret. == News == Forward anonymity, along with other privacy-protecting measures, received a burst of media attention after the leak of classified information by Edward Snowden, beginning in June, 2013, which indicated that the NSA and FBI, through specially crafted backdoors in software and computer systems, were conducting mass surveillance over large parts of the population of both the United States (see Mass surveillance in the United States), Europe, Asia, and other parts of the world. They justified this practice as an aid to catch predatory pedophiles. Opponents to this practice argue that leaving in a back door to law enforcement increases the risk of attackers being able to decrypt information, as well as questioning its legality under the US Constitution, specifically being a form of illegal Search and Seizure.

  Read more →
• 

  Ordered dithering

  Ordered dithering is any image dithering algorithm which uses a pre-set threshold map tiled across an image. It is commonly used to display a continuous image on a display of smaller color depth. For example, Microsoft Windows uses it in 16-color graphics modes. With the most common "Bayer" threshold map, the algorithm is characterized by noticeable crosshatch patterns in the result. == Threshold map == The algorithm reduces the number of colors by applying a threshold map M to the pixels displayed, causing some pixels to change color, depending on the distance of the original color from the available color entries in the reduced palette. The first threshold maps were designed by hand to minimise the perceptual difference between a grayscale image and its two-bit quantisation for up to a 4x4 matrix. An optimal threshold matrix is one that for any possible quantisation of color has the minimum possible texture so that the greatest impression of the underlying feature comes from the image being quantised. It can be proven that for matrices whose side length is a power of two there is an optimal threshold matrix. The map may be rotated or mirrored without affecting the effectiveness of the algorithm. This threshold map (for sides with length as power of two) is also known as a Bayer matrix or, when unscaled, an index matrix. For threshold maps whose dimensions are a power of two, the map can be generated recursively via: M 2 n = 1 ( 2 n ) 2 [ 4 M n 4 M n + 2 J n 4 M n + 3 J n 4 M n + J n ] = J 2 ⊗ M n + 1 n 2 M 2 ⊗ J n , {\displaystyle \mathbf {M} _{2n}={\frac {1}{(2n)^{2}}}{\begin{bmatrix}4\mathbf {M} _{n}&4\mathbf {M} _{n}+2\mathbf {J} _{n}\\4\mathbf {M} _{n}+3\mathbf {J} _{n}&4\mathbf {M} _{n}+\mathbf {J} _{n}\end{bmatrix}}=\mathbf {J} _{2}\otimes \mathbf {M} _{n}+{\frac {1}{n^{2}}}\mathbf {M} _{2}\otimes \mathbf {J} _{n},} where J n {\displaystyle \mathbf {J} _{n}} are n × n {\displaystyle n\times n} matrices of ones and ⊗ {\displaystyle \otimes } is the Kronecker product. While the metric for texture that Bayer proposed could be used to find optimal matrices for sizes that are not a power of two, such matrices are uncommon as no simple formula for finding them exists, and relatively small matrix sizes frequently give excellent practical results (especially when combined with other modifications to the dithering algorithm). This function can also be expressed using only bit arithmetic: M(i, j) = bit_reverse(bit_interleave(bitwise_xor(i, j), i)) / n ^ 2 == Pre-calculated threshold maps == Rather than storing the threshold map as a matrix of n {\displaystyle n} × n {\displaystyle n} integers from 0 to n 2 {\displaystyle n^{2}} , depending on the exact hardware used to perform the dithering, it may be beneficial to pre-calculate the thresholds of the map into a floating point format, rather than the traditional integer matrix format shown above. For this, the following formula can be used: Mpre(i,j) = Mint(i,j) / n^2 This generates a standard threshold matrix. for the 2×2 map: this creates the pre-calculated map: Additionally, normalizing the values to average out their sum to 0 (as done in the dithering algorithm shown below) can be done during pre-processing as well by subtracting 1⁄2 of the largest value from every value: Mpre(i,j) = Mint(i,j) / n^2 – 0.5 maxValue creating the pre-calculated map: == Algorithm == The ordered dithering algorithm renders the image normally, but for each pixel, it offsets its color value with a corresponding value from the threshold map according to its location, causing the pixel's value to be quantized to a different color if it exceeds the threshold. For most dithering purposes, it is sufficient to simply add the threshold value to every pixel (without performing normalization by subtracting 1⁄2), or equivalently, to compare the pixel's value to the threshold: if the brightness value of a pixel is less than the number in the corresponding cell of the matrix, plot that pixel black, otherwise, plot it white. This lack of normalization slightly increases the average brightness of the image, and causes almost-white pixels to not be dithered. This is not a problem when using a gray scale palette (or any palette where the relative color distances are (nearly) constant), and it is often even desired, since the human eye perceives differences in darker colors more accurately than lighter ones, however, it produces incorrect results especially when using a small or arbitrary palette, so proper normalization should be preferred. In other words, the algorithm performs the following transformation on each color c of every pixel: c ′ = n e a r e s t _ p a l e t t e _ c o l o r ( c + r × ( M ( x mod n , y mod n ) − 1 / 2 ) ) {\displaystyle c'=\mathrm {nearest\_palette\_color} {\mathopen {}}\left(c+r\times \left(M(x{\bmod {n}},y{\bmod {n}})-1/2\right){\mathclose {}}\right)} where M(i, j) is the threshold map on the i-th row and j-th column, c′ is the transformed color, and r is the amount of spread in color space. Assuming an RGB palette with 23N evenly distanced colors where each color (a triple of red, green and blue values) is represented by an octet from 0 to 255, one would typically choose r ≈ 255 N {\textstyle r\approx {\frac {255}{N}}} . (1⁄2 is again the normalizing term.) Because the algorithm operates on single pixels and has no conditional statements, it is very fast and suitable for real-time transformations. Additionally, because the location of the dithering patterns always stays the same relative to the display frame, it is less prone to jitter than error-diffusion methods, making it suitable for animations. Because the patterns are more repetitive than error-diffusion method, an image with ordered dithering compresses better. Ordered dithering is more suitable for line-art graphics as it will result in straighter lines and fewer anomalies. The values read from the threshold map should preferably scale into the same range as the minimal difference between distinct colors in the target palette. Equivalently, the size of the map selected should be equal to or larger than the ratio of source colors to target colors. For example, when quantizing a 24 bpp image to 15 bpp (256 colors per channel to 32 colors per channel), the smallest map one would choose would be 4×2, for the ratio of 8 (256:32). This allows expressing each distinct tone of the input with different dithering patterns. === A variable palette: pattern dithering === == Non-Bayer approaches == The above thresholding matrix approach describes the Bayer family of ordered dithering algorithms. A number of other algorithms are also known; they generally involve changes in the threshold matrix, which changes the distribution of the "noise" introduced by all kinds of dithering (the difference between the original image and the dithered image). === Halftone === Halftone dithering performs a form of clustered dithering, creating a look similar to halftone patterns, using a specially crafted matrix. === Void and cluster === The Void and cluster algorithm uses a pre-generated blue noise as the matrix for the dithering process. The blue noise matrix keeps the Bayer's good high frequency content, but with a more uniform coverage of all the frequencies involved shows a much lower amount of patterning. The "voids-and-cluster" method gets its name from the matrix generation procedure, where a black image with randomly initialized white pixels is gaussian-blurred to find the brightest and darkest parts, corresponding to voids and clusters. After a few swaps have evenly distributed the bright and dark parts, the pixels are numbered by importance. It takes significant computational resources to generate the blue noise matrix: on a modern computer a 64×64 matrix requires a couple seconds using the original algorithm. This algorithm can be extended to make animated dither masks which also consider the axis of time. This is done by running the algorithm in three dimensions and using a kernel which is a product of a two-dimensional gaussian kernel on the XY plane, and a one-dimensional Gaussian kernel on the Z axis. === Simulated Annealing === Simulated annealing can generate dither masks by starting with a flat histogram and swapping values to optimize a loss function. The loss function controls the spectral properties of the mask, allowing it to make blue noise or noise patterns meant to be filtered by specific filters. The algorithm can also be extended over time for animated dither masks with chosen temporal properties.

  Read more →
• 

  Backup

  In information technology, a backup, or data backup is a copy of computer data taken and stored elsewhere so that it may be used to restore the original after a data loss event. The verb form, referring to the process of doing so, is "back up", whereas the noun and adjective form is "backup". Backups can be used to recover data after its loss from data deletion or corruption, or to recover data from an earlier time. Backups provide a simple form of IT disaster recovery; however not all backup systems are able to reconstitute a computer system or other complex configuration such as a computer cluster, active directory server, or database server. A backup system contains at least one copy of all data considered worth saving. The data storage requirements can be large. An information repository model may be used to provide structure to this storage. There are different types of data storage devices used for copying backups of data that is already in secondary storage onto archive files. There are also different ways these devices can be arranged to provide geographic dispersion, data security, and portability. Data is selected, extracted, and manipulated for storage. The process can include methods for dealing with live data, including open files, as well as compression, encryption, and de-duplication. Additional techniques apply to enterprise client-server backup. Backup schemes may include dry runs that validate the reliability of the data being backed up. There are limitations and human factors involved in any backup scheme. == Storage == A backup strategy requires an information repository, "a secondary storage space for data" that aggregates backups of data "sources". The repository could be as simple as a list of all backup media (DVDs, etc.) and the dates produced, or could include a computerized index, catalog, or relational database. === 3-2-1 Backup Rule === The backup data needs to be stored, requiring a backup rotation scheme, which is a system of backing up data to computer media that limits the number of backups of different dates retained separately, by appropriate re-use of the data storage media by overwriting of backups no longer needed. The scheme determines how and when each piece of removable storage is used for a backup operation and how long it is retained once it has backup data stored on it. The 3-2-1 rule can aid in the backup process. It states that there should be at least 3 copies of the data, stored on 2 different types of storage media, and one copy should be kept offsite, in a remote location (this can include cloud storage). 2 or more different media should be used to eliminate data loss due to similar reasons (for example, optical discs may tolerate being underwater while LTO tapes may not, and SSDs cannot fail due to head crashes or damaged spindle motors since they do not have any moving parts, unlike hard drives). An offsite copy protects against fire, theft of physical media (such as tapes or discs) and natural disasters like floods and earthquakes. Physically protected hard drives are an alternative to an offsite copy, but they have limitations like only being able to resist fire for a limited period of time, so an offsite copy still remains as the ideal choice. Because there is no perfect storage, many backup experts recommend maintaining a second copy on a local physical device, even if the data is also backed up offsite. === Backup methods === ==== Unstructured ==== An unstructured repository may simply be a stack of tapes, DVD-Rs or external HDDs with minimal information about what was backed up and when. This method is the easiest to implement, but unlikely to achieve a high level of recoverability as it lacks automation. ==== Full only/System imaging ==== A repository using this backup method contains complete source data copies taken at one or more specific points in time. Copying system images, this method is frequently used by computer technicians to record known good configurations. However, imaging is generally more useful as a way of deploying a standard configuration to many systems rather than as a tool for making ongoing backups of diverse systems. ==== Incremental ==== An incremental backup stores data changed since a reference point in time. Duplicate copies of unchanged data are not copied. Typically a full backup of all files is made once or at infrequent intervals, serving as the reference point for an incremental repository. Subsequently, a number of incremental backups are made after successive time periods. Restores begin with the last full backup and then apply the incrementals. Some backup systems can create a synthetic full backup from a series of incrementals, thus providing the equivalent of frequently doing a full backup. When done to modify a single archive file, this speeds restores of recent versions of files. ==== Near-CDP ==== Continuous Data Protection (CDP) refers to a backup that instantly saves a copy of every change made to the data. This allows restoration of data to any point in time and is the most comprehensive and advanced data protection. Near-CDP backup applications—often marketed as "CDP"—automatically take incremental backups at a specific interval, for example every 15 minutes, one hour, or 24 hours. They can therefore only allow restores to an interval boundary. Near-CDP backup applications use journaling and are typically based on periodic "snapshots", read-only copies of the data frozen at a particular point in time. Near-CDP (except for Apple Time Machine) intent-logs every change on the host system, often by saving byte or block-level differences rather than file-level differences. This backup method differs from simple disk mirroring in that it enables a roll-back of the log and thus a restoration of old images of data. Intent-logging allows precautions for the consistency of live data, protecting self-consistent files but requiring applications "be quiesced and made ready for backup." Near-CDP is more practicable for ordinary personal backup applications, as opposed to true CDP, which must be run in conjunction with a virtual machine or equivalent and is therefore generally used in enterprise client-server backups. Software may create copies of individual files such as written documents, multimedia projects, or user preferences, to prevent failed write events caused by power outages, operating system crashes, or exhausted disk space, from causing data loss. A common implementation is an appended ".bak" extension to the file name. ==== Reverse incremental ==== A Reverse incremental backup method stores a recent archive file "mirror" of the source data and a series of differences between the "mirror" in its current state and its previous states. A reverse incremental backup method starts with a non-image full backup. After the full backup is performed, the system periodically synchronizes the full backup with the live copy, while storing the data necessary to reconstruct older versions. This can either be done using hard links—as Apple Time Machine does, or using binary diffs. ==== Differential ==== A differential backup saves only the data that has changed since the last full backup. This means a maximum of two backups from the repository are used to restore the data. However, as time from the last full backup (and thus the accumulated changes in data) increases, so does the time to perform the differential backup. Restoring an entire system requires starting from the most recent full backup and then applying just the last differential backup. A differential backup copies files that have been created or changed since the last full backup, regardless of whether any other differential backups have been made since, whereas an incremental backup copies files that have been created or changed since the most recent backup of any type (full or incremental). Changes in files may be detected through a more recent date/time of last modification file attribute, and/or changes in file size. Other variations of incremental backup include multi-level incrementals and block-level incrementals that compare parts of files instead of just entire files. === Storage media === Regardless of the repository model that is used, the data has to be copied onto an archive file data storage medium. The medium used is also referred to as the type of backup destination. ==== Magnetic tape ==== Magnetic tape was for a long time the most commonly used medium for bulk data storage, backup, archiving, and interchange. It was previously a less expensive option, but this is no longer the case for smaller amounts of data. Tape is a sequential access medium, so the rate of continuously writing or reading data can be very fast. While tape media itself has a low cost per space, tape drives are typically dozens of times as expensive as hard disk drives and optical drives. Tape media are generally rotated on a schedule so at least one set is off-site in case something should happe

  Read more →
• 

  Business intelligence

  Business intelligence (BI) consists of strategies, methodologies, and technologies used by enterprises for data analysis and management of business information to inform business strategies and business operations. Common functions of BI technologies include reporting, online analytical processing, analytics, dashboard development, data mining, process mining, complex event processing, business performance management, benchmarking, text mining, predictive analytics, and prescriptive analytics. BI tools can handle large amounts of structured and sometimes unstructured data to help organizations identify, develop, and otherwise create new strategic business opportunities. They aim to allow for the easy interpretation of these big data. Identifying new opportunities and implementing an effective strategy based on insights is assumed to potentially provide businesses with a competitive market advantage and long-term stability, and help them take strategic decisions. Business intelligence can be used by enterprises to support a wide range of business decisions ranging from operational to strategic. Basic operating decisions include product positioning or pricing. Strategic business decisions involve priorities, goals, and directions at the broadest level. In all cases, business intelligence is considered most effective when it combines data from the market in which a company operates (external data) with data from internal company sources, such as financial and operational information. When integrated, external and internal data provide a comprehensive view that creates ‘intelligence’ not possible from any single data source alone. Among their many uses, business intelligence tools empower organizations to gain insight into new markets, to assess demand and suitability of products and services for different market segments, and to gauge the impact of marketing efforts. BI applications use data gathered from a data warehouse (DW) or from a data mart, and the concepts of BI and DW combine as "BI/DW" or as "BIDW". A data warehouse contains a copy of analytical data that facilitates decision support. == History == The earliest known use of the term business intelligence is in Richard Millar Devens' Cyclopædia of Commercial and Business Anecdotes (1865). Devens used the term to describe how the banker Sir Henry Furnese gained profit by receiving and acting upon information about his environment, prior to his competitors: Throughout Holland, Flanders, France, and Germany, he maintained a complete and perfect train of business intelligence. The news of the many battles fought was thus received first by him, and the fall of Namur added to his profits, owing to his early receipt of the news. The ability to collect and react accordingly based on the information retrieved, Devens says, is central to business intelligence. When Hans Peter Luhn, a researcher at IBM, used the term business intelligence in an article published in 1958, he employed the Webster's Dictionary definition of intelligence: "the ability to apprehend the interrelationships of presented facts in such a way as to guide action towards a desired goal." In 1989, Howard Dresner (later a Gartner analyst) proposed business intelligence as an umbrella term to describe "concepts and methods to improve business decision making by using fact-based support systems." It was not until the late 1990s that this usage was widespread. == Definition == According to Solomon Negash and Paul Gray, business intelligence (BI) can be defined as systems that combine: Data gathering Data storage Knowledge management with analysis to evaluate complex corporate and competitive information for presentation to planners and decision makers, with the objective of improving the timeliness and the quality of the input to the decision process." According to Forrester Research, business intelligence is "a set of methodologies, processes, architectures, and technologies that transform raw data into meaningful and useful information used to enable more effective strategic, tactical, and operational insights and decision-making." Under this definition, business intelligence encompasses information management (data integration, data quality, data warehousing, master-data management, text- and content-analytics, et al.). Therefore, Forrester refers to data preparation and data usage as two separate but closely linked segments of the business-intelligence architectural stack. Some elements of business intelligence are: Multidimensional aggregation and allocation Denormalization, tagging, and standardization Realtime reporting with analytical alert A method of interfacing with unstructured data sources Group consolidation, budgeting, and rolling forecasts Statistical inference and probabilistic simulation Key performance indicators optimization Version control and process management Open item management Forrester distinguishes this from the business-intelligence market, which is "just the top layers of the BI architectural stack, such as reporting, analytics, and dashboards." === Compared with competitive intelligence === Though the term business intelligence is sometimes a synonym for competitive intelligence (because they both support decision making), BI uses technologies, processes, and applications to analyze mostly internal, structured data and business processes while competitive intelligence gathers, analyzes, and disseminates information with a topical focus on company competitors. If understood broadly, competitive intelligence can be considered as a subset of business intelligence. === Compared with business analytics === Business intelligence and business analytics are sometimes used interchangeably, but there are alternate definitions. Thomas Davenport, professor of information technology and management at Babson College argues that business intelligence should be divided into querying, reporting, Online analytical processing (OLAP), an "alerts" tool, and business analytics. In this definition, business analytics is the subset of BI focusing on statistics, prediction, and optimization, rather than the reporting functionality. == Unstructured data == Business operations can generate a very large amount of data in the form of emails, memos, notes from call centers, news, user groups, chats, reports, web pages, presentations, image files, video files, and marketing material. According to Merrill Lynch, more than 85% of all business information exists in these forms; a company might only use such a document a single time. Because of the way it is produced and stored, this information is either unstructured or semi-structured. The management of semi-structured data is an unsolved problem in the information technology industry. According to projections from Gartner (2003), white-collar workers spend 30–40% of their time searching, finding, and assessing unstructured data. BI uses both structured and unstructured data. The former is easy to search, and the latter contains a large quantity of the information needed for analysis and decision-making. Because of the difficulty of properly searching, finding, and assessing unstructured or semi-structured data, organizations may not draw upon these vast reservoirs of information, which could influence a particular decision, task, or project. This can ultimately lead to poorly informed decision-making. Therefore, when designing a business intelligence/DW solution, the specific problems associated with semi-structured and unstructured data must be accommodated, as well as those associated with structured data. === Limitations of semi-structured and unstructured data === There are several challenges to developing BI with semi-structured data. According to Inmon & Nesavich, some of those are: Physically accessing unstructured textual data – unstructured data is stored in a huge variety of formats. Terminology – Among researchers and analysts, there is a need to develop standardized terminology. Volume of data – As stated earlier, up to 85% of all data exists as semi-structured data. Couple that with the need for word-to-word and semantic analysis. Searchability of unstructured textual data – A simple search on some data, e.g. apple, results in links where there is a reference to that precise search term. (Inmon & Nesavich, 2008) gives an example: "a search is made on the term felony. In a simple search, the term felony is used, and everywhere there is a reference to felony, a hit to an unstructured document is made. But a simple search is crude. It does not find references to crime, arson, murder, embezzlement, vehicular homicide, and such, even though these crimes are types of felonies". === Metadata === To solve problems with searchability and assessment of data, it is necessary to know something about the content. This can be done by adding context through the use of metadata. Many systems already capture some metadata (e.g. filename, author, size, etc.), but more usef

  Read more →
• 

  Smart-ID

  Smart-ID is an electronic authentication tool developed by SK ID Solutions, an Estonian company. Users can log in to various electronic services and sign documents with an electronic signature. Smart-ID meets the European Union's eIDAS Regulation and the European Central Bank's standards for a secure authentication solution. Smart-ID is a Qualified Signature Creator Device (QSCD) that can issue a Qualified Electronic Signature (QES). The Smart-ID app is compatible with both iOS and Android devices and does not require a SIM card. By 2021, the Smart-ID application was launched in the Huawei AppGallery. As of May 2023, Smart-ID has 3,298,969 active users across the Baltic States (Latvia, Lithuania, and Estonia). Every month, the Smart-ID processes 79 million transactions. In March 2023, Smart-ID users made an exceptional 85 million transactions. == History == In November 2016, SK ID Solutions debuted the Smart-ID tool for the first time at its annual conference. In February 2017, eKool, Starman, and Tallinn Kaubamaja Grupp were the first to implement Smart-ID authentication in their e-services. In March 2017, Smart-ID was added as an authentication option to SEB bank and Swedbank's online banking in all three Baltic States. Dokobit, previously known as DigiDoc, began offering its clients the ability to use e-services using Smart-ID in April 2017. More than 100 service providers had implemented Smart-ID as an authentication solution for their services by November 2019. At its annual conference on November 8, 2018, SK ID Solutions revealed that Smart-ID had been certified as compatible with the QSCD[8] level, the highest level of qualified electronic signature in the European Union, following a rigorous certification process. As a result, the Smart-QES-level ID's electronic signature, the digital counterpart of a handwritten signature, is now available to all users who have registered with the tool. This signature is accepted by all European Union member states. On August 26, 2019, Estonian Information Systems Supervisory Authority experts reviewed Smart-ID (ISSA). Based on the methods provided in the eIDAS Regulation, the expert committee concluded that Smart-ID offers a high level of electronic identification assurance. SK ID Solutions and RIA struck an agreement in September 2019 that allows Smart-ID to authenticate Estonian state e-services via RIA's central authentication service, which is used by over 60 public authorities. Smart-ID accounts created three years ago have expired in January 2020. Therefore, renewing them and performing mandatory updates was necessary. In February 2020, SK ID Solutions announced that Smart-ID could be used to give digital signatures in the national digital signature software DigiDoc4, which up until this moment was only possible with ID cards via Mobile-ID. Users must have at least version 4.2.4.71 or later of the DigiDoc4 software installed on their computers to use this feature. Since February 2020, Smart-ID accounts can now be created with biometric information from an ID card or passport, but only by users who have previously used a Smart-ID account. Since October 2022, 13–17 years old minors in Lithuania are able to create a Smart-ID account using biometric information too. A parent or legal guardian must approve the registration. SK ID Solutions collaborated on the new solution with iProov from the United Kingdom and InnoValor from the Netherlands. TÜV Informationstechnik GmbH, a German certification company, assessed it. Since May 2023, Smart-ID can be used to submit company's annual reports in Estonia and digitally sign anything in the e-business register using your PIN2. == Overview == The Smart-ID app is available for download on Google Play and Apple's App Store. Android 4.4 and iOS 11 are the oldest supported operating system versions for Smart-ID. Smart-ID works on the premise of two-factor authentication, combining an intelligent device (something the user owns) with PINs (something the user knows). A new user must first authenticate themselves with an ID card or a mobile phone number and then confirm a PIN1 and PIN2 code, either manually or automatically produced. The first PIN is used to authenticate a person's identity when accessing e-banking or e-services, while the second PIN is used to support electronic signatures and authenticate transactions (e.g., transfers). The PIN1 code must be four digits long, while the PIN2 code must be five digits long. To log in to an e-service, the user must use Smart-ID as the authentication method and enter their unique Smart-ID user ID. A notification will open on the user's smart device where the software is installed and display a verification code. If the code matches the code presented to the user by the e-service, then the user can confirm the match by entering their PIN1 code. The user must verify the action with their PIN2 code when giving digital signatures. A Smart-ID account is valid for three years. The report can be updated, changed, and deleted at any given time, free of charge. Smart-ID is available in five languages: Estonian, Latvian, Lithuanian, Russian, and English. An international survey conducted in 2021 revealed that Smart-ID is the most reliable authentication solution in Baltic countries. In January 2023, the number of times Smart-ID was used to access State Authentication Service (TARA) in Estonia has surpassed those of Mobile-ID and ID-cards for the first time since July 2022. == Security == Smart-ID is based on Cybernetica's SplitKey authentication and digital signature platform technology, for which the company has filed a patent application. Public key cryptography, digital signature methods, and critical public infrastructures are all used in the technology. The user's PIN is not saved on the device and is only needed to decrypt the private key in the Smart-ID app. When the user inputs the PIN, the private key is cracked, and the answer is transmitted to the Smart-ID server, where a portion of the key given by the app is joined with the server's encrypted key. The app will block the user from accessing it for three hours if they input the incorrect PIN three times in a row. If this happens once again, the app will lock for 24 hours. If this happens a third time, the account will be permanently disabled. PINs cannot be changed or recovered once an account has been created. The user must create a new account if the account is permanently blocked. Smart-ID uses the Apple and Google messaging networks to notify the app when new data is saved on its servers. == Phishing == In February 2019, unknown criminals attempted to create Smart-ID accounts with stolen IDs obtained via phishing customers' text messages and website addresses, according to a monthly report by the Estonian Information System Manager in April 2019. The Latvian Information Technology Security Incident Assessment Body Cert was also notified of these intrusions on March 1. Fraudsters sent emails to potential victims pretending to be bank representatives. The mails linked users to a phishing page after redirecting them to a phony bank login page. Victims were asked to log in using their identification information and PIN1 code. The fraudsters then began the process of generating a new Smart-ID account. As a result, the victim had to input a PIN2 number, which permitted the fraudster to finish setting up a new tab with the victim's personal information. Fraudsters in Estonia were able to log in to multiple e-services utilizing Smart-ID using a Smart-ID account and the victim's data. On behalf of the victims, fraudsters also employed online banking services. Later, the Estonian Information System Manager identified several victims, some of whom had also experienced financial losses. The Estonian Information System Manager requested a full report on the event from SK ID Solutions. The organization opted not to criticize the corporation after receiving the information, although it did propose that the procedure of creating Smart-ID accounts be reviewed. According to the Estonian Banking Association, Estonian banks have not discontinued using Smart-ID and do not think it is required. Smart-ID was exposed to a thorough review process in September 2019 to determine this authentication instrument's level of security. Reviewers discovered no flaws, and SK ID Solutions and the Estonian Information System Manager signed a contract. Estonia later introduced Smart-ID and other authentication mechanisms to the central public services portal.

  Read more →
• 

  Model-based clustering

  In statistics, cluster analysis is the algorithmic grouping of objects into homogeneous groups based on numerical measurements. Model-based clustering based on a statistical model for the data, usually a mixture model. This has several advantages, including a principled statistical basis for clustering, and ways to choose the number of clusters, to choose the best clustering model, to assess the uncertainty of the clustering, and to identify outliers that do not belong to any group. == Model-based clustering == Suppose that for each of n {\displaystyle n} observations we have data on d {\displaystyle d} variables, denoted by y i = ( y i , 1 , … , y i , d ) {\displaystyle y_{i}=(y_{i,1},\ldots ,y_{i,d})} for observation i {\displaystyle i} . Then model-based clustering expresses the probability density function of y i {\displaystyle y_{i}} as a finite mixture, or weighted average of G {\displaystyle G} component probability density functions: p ( y i ) = ∑ g = 1 G τ g f g ( y i ∣ θ g ) , {\displaystyle p(y_{i})=\sum _{g=1}^{G}\tau _{g}f_{g}(y_{i}\mid \theta _{g}),} where f g {\displaystyle f_{g}} is a probability density function with parameter θ g {\displaystyle \theta _{g}} , τ g {\displaystyle \tau _{g}} is the corresponding mixture probability where ∑ g = 1 G τ g = 1 {\displaystyle \sum _{g=1}^{G}\tau _{g}=1} . Then in its simplest form, model-based clustering views each component of the mixture model as a cluster, estimates the model parameters, and assigns each observation to cluster corresponding to its most likely mixture component. === Gaussian mixture model === The most common model for continuous data is that f g {\displaystyle f_{g}} is a multivariate normal distribution with mean vector μ g {\displaystyle \mu _{g}} and covariance matrix Σ g {\displaystyle \Sigma _{g}} , so that θ g = ( μ g , Σ g ) {\displaystyle \theta _{g}=(\mu _{g},\Sigma _{g})} . This defines a Gaussian mixture model. The parameters of the model, τ g {\displaystyle \tau _{g}} and θ g {\displaystyle \theta _{g}} for g = 1 , … , G {\displaystyle g=1,\ldots ,G} , are typically estimated by maximum likelihood estimation using the expectation-maximization algorithm (EM); see also EM algorithm and GMM model. Bayesian inference is also often used for inference about finite mixture models. The Bayesian approach also allows for the case where the number of components, G {\displaystyle G} , is infinite, using a Dirichlet process prior, yielding a Dirichlet process mixture model for clustering. === Choosing the number of clusters === An advantage of model-based clustering is that it provides statistically principled ways to choose the number of clusters. Each different choice of the number of groups G {\displaystyle G} corresponds to a different mixture model. Then standard statistical model selection criteria such as the Bayesian information criterion (BIC) can be used to choose G {\displaystyle G} . The integrated completed likelihood (ICL) is a different criterion designed to choose the number of clusters rather than the number of mixture components in the model; these will often be different if highly non-Gaussian clusters are present. === Parsimonious Gaussian mixture model === For data with high dimension, d {\displaystyle d} , using a full covariance matrix for each mixture component requires estimation of many parameters, which can result in a loss of precision, generalizabity and interpretability. Thus it is common to use more parsimonious component covariance matrices exploiting their geometric interpretation. Gaussian clusters are ellipsoidal, with their volume, shape and orientation determined by the covariance matrix. Consider the eigendecomposition of a matrix Σ g = λ g D g A g D g T , {\displaystyle \Sigma _{g}=\lambda _{g}D_{g}A_{g}D_{g}^{T},} where D g {\displaystyle D_{g}} is the matrix of eigenvectors of Σ g {\displaystyle \Sigma _{g}} , A g = diag { A 1 , g , … , A d , g } {\displaystyle A_{g}={\mbox{diag}}\{A_{1,g},\ldots ,A_{d,g}\}} is a diagonal matrix whose elements are proportional to the eigenvalues of Σ g {\displaystyle \Sigma _{g}} in descending order, and λ g {\displaystyle \lambda _{g}} is the associated constant of proportionality. Then λ g {\displaystyle \lambda _{g}} controls the volume of the ellipsoid, A g {\displaystyle A_{g}} its shape, and D g {\displaystyle D_{g}} its orientation. Each of the volume, shape and orientation of the clusters can be constrained to be equal (E) or allowed to vary (V); the orientation can also be spherical, with identical eigenvalues (I). This yields 14 possible clustering models, shown in this table: It can be seen that many of these models are more parsimonious, with far fewer parameters than the unconstrained model that has 90 parameters when G = 4 {\displaystyle G=4} and d = 9 {\displaystyle d=9} . Several of these models correspond to well-known heuristic clustering methods. For example, k-means clustering is equivalent to estimation of the EII clustering model using the classification EM algorithm. The Bayesian information criterion (BIC) can be used to choose the best clustering model as well as the number of clusters. It can also be used as the basis for a method to choose the variables in the clustering model, eliminating variables that are not useful for clustering. Different Gaussian model-based clustering methods have been developed with an eye to handling high-dimensional data. These include the pgmm method, which is based on the mixture of factor analyzers model, and the HDclassif method, based on the idea of subspace clustering. The mixture-of-experts framework extends model-based clustering to include covariates. == Example == We illustrate the method with a dateset consisting of three measurements (glucose, insulin, sspg) on 145 subjects for the purpose of diagnosing diabetes and the type of diabetes present. The subjects were clinically classified into three groups: normal, chemical diabetes and overt diabetes, but we use this information only for evaluating clustering methods, not for classifying subjects. The BIC plot shows the BIC values for each combination of the number of clusters, G {\displaystyle G} , and the clustering model from the Table. Each curve corresponds to a different clustering model. The BIC favors 3 groups, which corresponds to the clinical assessment. It also favors the unconstrained covariance model, VVV. This fits the data well, because the normal patients have low values of both sspg and insulin, while the distributions of the chemical and overt diabetes groups are elongated, but in different directions. Thus the volumes, shapes and orientations of the three groups are clearly different, and so the unconstrained model is appropriate, as selected by the model-based clustering method. The classification plot shows the classification of the subjects by model-based clustering. The classification was quite accurate, with a 12% error rate as defined by the clinical classification. Other well-known clustering methods performed worse with higher error rates, such as single-linkage clustering with 46%, average link clustering with 30%, complete-linkage clustering also with 30%, and k-means clustering with 28%. == Outliers in clustering == An outlier in clustering is a data point that does not belong to any of the clusters. One way of modeling outliers in model-based clustering is to include an additional mixture component that is very dispersed, with for example a uniform distribution. Another approach is to replace the multivariate normal densities by t {\displaystyle t} -distributions, with the idea that the long tails of the t {\displaystyle t} -distribution would ensure robustness to outliers. However, this is not breakdown-robust. A third approach is the "tclust" or data trimming approach which excludes observations identified as outliers when estimating the model parameters. == Non-Gaussian clusters and merging == Sometimes one or more clusters deviate strongly from the Gaussian assumption. If a Gaussian mixture is fitted to such data, a strongly non-Gaussian cluster will often be represented by several mixture components rather than a single one. In that case, cluster merging can be used to find a better clustering. A different approach is to use mixtures of complex component densities to represent non-Gaussian clusters. == Non-continuous data == === Categorical data === Clustering multivariate categorical data is most often done using the latent class model. This assumes that the data arise from a finite mixture model, where within each cluster the variables are independent. === Mixed data === These arise when variables are of different types, such as continuous, categorical or ordinal data. A latent class model for mixed data assumes local independence between the variable. The location model relaxes the local independence assumption. The clustMD approach assumes that the observed variables are manifestations of underlying continuous Gaussian latent

  Read more →
• 

  Data hub

  A data hub is a center of data exchange that is supported by data science, data engineering, and data warehouse technologies to interact with endpoints such as applications and algorithms. == Features == A data hub differs from a data warehouse in that it is generally unintegrated and often at different grains. It differs from an operational data store because a data hub does not need to be limited to operational data. A data hub differs from a data lake by homogenizing data and possibly serving data in multiple desired formats, rather than simply storing it in one place, and by adding other value to the data such as de-duplication, quality, security, and a standardized set of query services. A data lake tends to store data in one place for availability, and allow/require the consumer to process or add value to the data. Data hubs are ideally the "go-to" place for data within an enterprise, so that many point-to-point connections between callers and data suppliers do not need to be made, and so that the data hub organization can negotiate deliverables and schedules with various data enclave teams, rather than being an organizational free-for-all as different teams try to get new services and features from many other teams.

  Read more →
• 

  G.hn

  Gigabit Home Networking (G.hn) is a specification for wired home networking that supports speeds up to 2 Gbit/s and operates over four types of legacy wires: telephone wiring, coaxial cables, power lines and plastic optical fiber. Some benefits of a multi-wire standard are lower equipment development costs and lower deployment costs for service providers (by allowing customer self-install). == History == G.hn was developed under the International Telecommunication Union's Telecommunication Standardization sector (the ITU-T) and promoted by the HomeGrid Forum and several other organizations. ITU-T Recommendation (the ITU's term for standard) G.9960, which received approval on October 9, 2009, specified the physical layers and the architecture of G.hn. The Data Link Layer (Recommendation G.9961) was approved on June 11, 2010. Prominent organizations, including CEPca, HomePNA, and UPA, who were creators of some of these interfaces, rallied behind the latest version of the standard, emphasizing its potential and significance in the home networking domain. Moreover, the ITU-T extended the technology with multiple input, multiple output (MIMO) technology to increase data rates and signaling distance. This new feature was approved in March 2012 under G.9963 Recommendation. The development and promotion of G.hn have been significantly supported by the HomeGrid Forum and several other organizations. The technology was not only designed to address home-networking challenges but also found applications beyond this initial scope, showcasing its versatility and potential in the networking domain. == Technical specifications == === Technical overview === G.hn specifies a single physical layer based on fast Fourier transform (FFT) orthogonal frequency-division multiplexing (OFDM) modulation and low-density parity-check code (LDPC) forward error correction (FEC) code. G.hn includes the capability to notch specific frequency bands to avoid interference with amateur radio bands and other licensed radio services. G.hn includes mechanisms to avoid interference with legacy home networking technologies and also with other wireline systems such as VDSL2 or other types of DSL used to access the home. OFDM systems split the transmitted signal into multiple orthogonal sub-carriers. In G.hn each one of the sub-carriers is modulated using QAM. The maximum QAM constellation supported by G.hn is 4096-QAM (12-bit QAM). The G.hn media access control is based on a time division multiple access (TDMA) architecture, in which a "domain master" schedules Transmission Opportunities (TXOPs) that can be used by one or more devices in the "domain". There are two types of TXOPs: Contention-Free Transmission Opportunities (CFTXOP), which have a fixed duration and are allocated to a specific pair of transmitter and receiver. CFTXOP are used for implementing TDMA Channel Access for specific applications that require quality of service (QoS) guarantees. Shared Transmission Opportunities (STXOP), which are shared among multiple devices in the network. STXOP are divided into Time Slots (TS). There are two types of TS: Contention-Free Time Slots (CFTS), which are used for implementing "implicit" token passing Channel Access. In G.hn, a series of consecutive CFTS is allocated to a number of devices. The allocation is performed by the "domain master" and broadcast to all nodes in the network. There are pre-defined rules that specify which device can transmit after another device has finished using the channel. As all devices know "who is next", there is no need to explicitly send a "token" between devices. The process of "passing the token" is implicit and ensures that there are no collisions during Channel access. Contention-Based Time Slots (CBTS), which are used for implementing CSMA/CARP Channel Access. In general, CSMA systems cannot completely avoid collisions, so CBTS are only useful for applications that do not have strict Quality of Service requirements. ==== Optimization for each medium ==== Although most elements of G.hn are common for all three media supported by the standard (power lines, phone lines and coaxial cable), G.hn includes media-specific optimizations for each media. Some of these media-specific parameters include: OFDM Carrier Spacing: 195.31 kHz in coaxial, 48.82 kHz in phone lines, 24.41 kHz in power lines. FEC Rates: G.hn's FEC can operate with code rates 1/2, 2/3, 5/6, 16/18 and 20/21. Although these rates are not media specific, it is expected that the higher code rates will be used in cleaner media (such as coaxial) while the lower code rates will be used in noisy environments such as power lines. Automatic repeat request (ARQ) mechanisms: G.hn supports operation both with and without ARQ (re-transmission). Although this is not media specific, it is expected that ARQ-less operation is sometimes appropriate for cleaner media (such as coaxial) while ARQ operation is appropriate for noisy environments such as power lines. Power levels and frequency bands: G.hn defines different power masks for each medium. MIMO support: Recommendation G.9963 includes provisions for transmitting G.hn signals over multiple AC wires (phase, neutral, ground), if they are physically available. In July 2016, G.9963 was updated to include MIMO support over twisted pairs. ==== Security ==== G.hn uses the Advanced Encryption Standard (AES) encryption algorithm (with a 128-bit key length) using the CCMP protocol to ensure confidentiality and message integrity. Authentication and key exchange is done following ITU-T Recommendation X.1035. G.hn specifies point-to-point security inside a domain, which means that each pair of transmitter and receiver uses a unique encryption key which is not shared by other devices in the same domain. For example, if node Alice sends data to node Bob, node Eve (in the same domain as Alice and Bob) will not be able to easily eavesdrop their communication. G.hn supports the concept of relays, in which one device can receive a message from one node and deliver it to another node farther away in the same domain. Relaying becomes critical for applications with complex network topologies that need to cover large distances, such as those found in industrial or utility applications. While a relay can read the source and target addresses, it cannot read the message's content due to its body being end-to-end-encrypted. ==== Profiles ==== The G.hn architecture includes the concept of profiles. Profiles are intended to address G.hn nodes with significantly different levels of complexity. In G.hn the higher complexity profiles are proper supersets of lower complexity profiles, so that devices based on different profiles can interoperate with each other. Examples of G.hn devices based on high complexity profiles are Residential Gateways or Set-Top Boxes. Examples of G.hn devices based on low complexity profiles are home automation, home security and smart grid devices. ==== Technical parameters ==== The chart depicts a summary of the crucial technical specifications of the G.hn standard. Many of these technical elements are consistent across different physical media, with variations seen in areas such as Tone Spacing and frequency ranges. This uniformity is essential as it allows silicon manufacturers to produce a singular chip capable of implementing all three media types, leading to cost savings. Presently, G.hn chipsets are compatible with all three media types. This compatibility allows system manufacturers to create devices that can adjust to any wiring type simply by modifying a software configuration in the equipment. === Spectrum === The G.hn spectrum depends on the medium as shown in the diagram below: === Protocol stack === G.hn specifies the physical layer and the data link layer, according to the OSI model. The G.hn Data Link Layer (Recommendation G.9961) is divided into three sub-layers: The Application Protocol Convergence (APC) Layer, which accepts frames (usually in Ethernet format) from the upper layer (Application Entity) and encapsulates them into G.hn APC protocol data units (APDUs). The maximum payload of each APDU is 214 bytes. The logical link control (LLC), which is responsible for encryption, aggregation, segmentation and automatic repeat-request. This sub-layer is also responsible for "relaying" of APDUs between nodes that may not be able to communicate through a direct connection. The medium access control (MAC), which schedules channel access. The G.hn physical layer (Recommendation G.9960) is divided into three sub-layers: The Physical Coding Sub-layer (PCS), responsible for generating PHY headers. The Physical Medium Attachment (PMA), responsible for scrambling and forward error correction coding/decoding. The Physical Medium Dependent (PMD), responsible for bit-loading and OFDM modulation. The interface between the Application Entity and the Data Link Layer is called A-interface. The interface between the Data Link Layer and the ph

  Read more →