AI For Business Development

AI For Business Development — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.

• 

  Celia (virtual assistant)

  Celia is an artificially intelligent virtual assistant developed by Huawei for their latest HarmonyOS and Android-based EMUI smartphones that lack Google Services and a Google Assistant. The assistant can perform day-to-day tasks, which include making a phone call, setting a reminder and checking the weather. It was unveiled on 7 April 2020 and got publicly released on 27 April 2020 via an OTA update solely to selected devices that can update their software to EMUI 10.1. Huawei had initially referred to the new assistant in late 2019 by having announced that there would be an English version of their already 2018 Chinese speaker assistant—Xiaoyi—to be released into the European markets. Due to the on-going China–United States trade war, the company's newly released smartphones were left without any Google services, including the loss of Google Assistant. This subsequently led to the development and release of Celia. AI technology is integrated into the software of Celia, which allows it to translate text using a phones camera and to identify everyday objects — similar to that of Google Lens. == Features == Celia has many features that are similar to that of its rivals: the Google Assistant and Siri. It can be triggered by the words, 'Hey Celia' or be summoned by pressing and holding down on the power button. The default search engine for Celia is Bing, but this can be changed in settings. Celia can make calls, check the agenda, send a message, show the weather, set alarms and control home appliances. The assistant also has the ability to integrate itself with the stock apps of the EMUI software and toggle with the device's settings, such as by turning on the flashlight and playing multimedia content, but with the users command. With the AI that is installed in Celia, it can identify food, everyday objects and translate text using the phones camera. In China, Chinese Xiaoyi packs with an LLM model called PanGu-Σ 3.0 AI on HarmonyOS 4.0 major upgrade improvements from Celia, making the assistant smarter and more advanced compared to when it was launched in 2020 on EMUI handsets in China and internationally, surpassing Apple and Google by the being the first in the AI industry, with a dedicated AI system framework of APIs on the latest operating system that evolves to a complete large dedicated AI software stack called Harmony Intelligence of Pangu Embedded variant model and MindSpore AI framework with Neural Network Runtime on OpenHarmony-based HarmonyOS NEXT base system to replace the dual framework system with a single frame HarmonyOS 5.0 version by Q4 2024, first introduced on June 21, 2024, in Developer Beta 1 preview release at HDC 2024. == Availability by country and language == Currently, Celia is available only in German, English, French and Spanish, and has been released in Germany, the UK, France, Spain, Chile, Mexico and Colombia. Huawei has said, that there will be more regions and languages to come. == Compatible devices == Celia only became available with the EMUI 10.1 update that was released in April, which means that a limited number of devices are compatible with it. More devices will be added to the list throughout the coming months as Celia's availability increases. The current list is shown below: === Huawei P series === Huawei P50 (Pro) Huawei P40 (Lite, Pro & Pro+) Huawei P30 (Pro) === Huawei Mate series === Huawei Mate 40 Huawei Mate 30 (Lite, Pro & RS Porche Design) Huawei MatePad Pro Huawei Mate 20 (Pro, 20X 4G, 20X 5G and RS Porche Design) Huawei Mate X & Xs === Huawei Nova series === Huawei Nova 6 (Nova 6 5G & Nova 6 SE) Huawei Nova 5 (Nova 5 Pro, Nova 5i Pro & Nova 5Z) Huawei Nova Y60 === Huawei Enjoy series === Huawei Enjoy 10S == Issues == Technology news website Engadget has noted that when saying, 'Hey Celia', out aloud in the presence of an iPhone, Siri will respond along with Celia; this is apparently because 'Celia' sounds similar to 'Siri'.

  Read more →
• 

  Hyper-encryption

  Hyper-encryption is a form of encryption invented by Michael O. Rabin which uses a high-bandwidth source of public random bits, together with a secret key that is shared by only the sender and recipient(s) of the message. It uses the assumptions of Ueli Maurer's bounded-storage model as the basis of its secrecy. Although everyone can see the data, decryption by adversaries without the secret key is still not feasible, because of the space limitations of storing enough data to mount an attack against the system. Unlike almost all other cryptosystems except the one-time pad, hyper-encryption can be proved to be information-theoretically secure, provided the storage bound cannot be surpassed. Moreover, if the necessary public information cannot be stored at the time of transmission, the plaintext can be shown to be impossible to recover, regardless of the computational capacity available to an adversary in the future, even if they have access to the secret key at that future time. A highly energy-efficient implementation of a hyper-encryption chip was demonstrated by Krishna Palem et al. using the Probabilistic CMOS or PCMOS technology and was shown to be ~205 times more efficient in terms of Energy-Performance-Product.

  Read more →
• 

  InfiniBand

  InfiniBand (IB) is a computer networking standard used in high-performance computing that features very high throughput and very low latency. It is used for data interconnect both among and within computers. InfiniBand is also used as either a direct or switched interconnect between servers and storage systems, as well as an interconnect between storage systems. It is designed to be scalable and uses a switched fabric network topology. Between 2014 and June 2016, it was the most commonly used interconnect in the TOP500 list of supercomputers. Mellanox (acquired by Nvidia) manufactures InfiniBand host bus adapters and network switches, which are used by large computer system and database vendors in their product lines. As a computer cluster interconnect, IB competes with Ethernet, Fibre Channel, and Intel Omni-Path. The technology is promoted by the InfiniBand Trade Association. == History == InfiniBand originated in 1999 from the merger of two competing designs: Future I/O and Next Generation I/O (NGIO). NGIO was led by Intel, with a specification released in 1998, and joined by Sun Microsystems and Dell. Future I/O was backed by Compaq, IBM, and Hewlett-Packard. This led to the formation of the InfiniBand Trade Association (IBTA), which included both sets of hardware vendors as well as software vendors such as Microsoft. At the time it was thought some of the more powerful computers were approaching the interconnect bottleneck of the PCI bus, in spite of upgrades like PCI-X. Version 1.0 of the InfiniBand Architecture Specification was released in 2000. Initially the IBTA vision for IB was simultaneously a replacement for PCI in I/O, Ethernet in the machine room, cluster interconnect and Fibre Channel. IBTA also envisaged decomposing server hardware on an IB fabric. Mellanox had been founded in 1999 to develop NGIO technology, but by 2001 shipped an InfiniBand product line called InfiniBridge at 10 Gbit/second speeds. Following the burst of the dot-com bubble there was hesitation in the industry to invest in such a far-reaching technology jump. By 2002, Intel announced that instead of shipping IB integrated circuits ("chips"), it would focus on developing PCI Express, and Microsoft discontinued IB development in favor of extending Ethernet. Sun Microsystems and Hitachi continued to support IB. In 2003, the System X supercomputer built at Virginia Tech used InfiniBand in what was estimated to be the third largest computer in the world at the time. The OpenIB Alliance (later renamed OpenFabrics Alliance) was founded in 2004 to develop an open set of software for the Linux kernel. By February, 2005, the support was accepted into the 2.6.11 Linux kernel. In November 2005 storage devices finally were released using InfiniBand from vendors such as Engenio. Cisco, desiring to keep technology superior to Ethernet off the market, adopted a "buy to kill" strategy. Cisco successfully killed InfiniBand switching companies such as Topspin via acquisition. Of the top 500 supercomputers in 2009, Gigabit Ethernet was the internal interconnect technology in 259 installations, compared with 181 using InfiniBand. In 2010, market leaders Mellanox and Voltaire merged, leaving just one other IB vendor, QLogic, primarily a Fibre Channel vendor. At the 2011 International Supercomputing Conference, links running at about 56 gigabits per second (known as FDR, see below), were announced and demonstrated by connecting booths in the trade show. In 2012, Intel acquired QLogic's InfiniBand technology, leaving only one independent supplier. By 2014, InfiniBand was the most popular internal connection technology for supercomputers, although within two years, 10 Gigabit Ethernet started displacing it. In 2016, it was reported that Oracle Corporation (an investor in Mellanox) might engineer its own InfiniBand hardware. In 2019 Nvidia acquired Mellanox, the last independent supplier of InfiniBand products. == Specification == Specifications are published by the InfiniBand trade association. === Performance === Original names for speeds were single-data rate (SDR), double-data rate (DDR) and quad-data rate (QDR) as given below. Subsequently, other three-letter initialisms were added for even higher data rates. Notes Each link is duplex. Links can be aggregated: most systems use a 4 link/lane connector (QSFP). HDR often makes use of 2x links (aka HDR100, 100 Gb link using 2 lanes of HDR, while still using a QSFP connector). NDR introduced OSFP connectors which host one or two links at 2x (NDR200) or 4x (NDR400). They are not logically configured as a single 8x link, even when connecting switches together with an OSFP cable. InfiniBand provides remote direct memory access (RDMA) capabilities for low CPU overhead. === Topology === InfiniBand uses a switched fabric topology, as opposed to early shared medium Ethernet. All transmissions begin or end at a channel adapter. Each processor contains a host channel adapter (HCA) and each peripheral has a target channel adapter (TCA). These adapters can also exchange information for security or quality of service (QoS). === Messages === InfiniBand transmits data in packets of up to 4 KB that are taken together to form a message. A message can be: a remote direct memory access read or write a channel send or receive a transaction-based operation (that can be reversed) a multicast transmission an atomic operation === Physical interconnection === In addition to a board form factor connection, it can use both active and passive copper (up to 10 meters) and optical fiber cable (up to 10 km). QSFP connectors are used. The InfiniBand Association also specified the CXP connector system for speeds up to 120 Gbit/s over copper, active optical cables, and optical transceivers using parallel multi-mode fiber cables with 24-fiber MPO connectors. === Software interfaces === Mellanox operating system support is available for Solaris, FreeBSD, Red Hat Enterprise Linux, SUSE Linux Enterprise Server (SLES), Windows, HP-UX, VMware ESX, and AIX. InfiniBand has no specific standard application programming interface (API). The standard only lists a set of verbs such as ibv_open_device or ibv_post_send, which are abstract representations of functions or methods that must exist. The syntax of these functions is left to the vendors. Sometimes for reference this is called the verbs API. The de facto standard software is developed by OpenFabrics Alliance and called the Open Fabrics Enterprise Distribution (OFED). It is released under two licenses GPL2 or BSD license for Linux and FreeBSD, and as Mellanox OFED for Windows (product names: WinOF / WinOF-2; attributed as host controller driver for matching specific ConnectX 3 to 5 devices) under a choice of BSD license for Windows. It has been adopted by most of the InfiniBand vendors, for Linux, FreeBSD, and Microsoft Windows. IBM refers to a software library called libibverbs, for its AIX operating system, as well as "AIX InfiniBand verbs". The Linux kernel support was integrated in 2005 into the kernel version 2.6.11. === Ethernet over InfiniBand === Ethernet over InfiniBand, abbreviated to EoIB, is an Ethernet implementation over the InfiniBand protocol and connector technology. EoIB enables multiple Ethernet bandwidths varying on the InfiniBand (IB) version. Ethernet's implementation of the Internet Protocol Suite, usually referred to as TCP/IP, is different in some details compared to the direct InfiniBand protocol in IP over IB (IPoIB).

  Read more →
• 

  Superincreasing sequence

  In mathematics, a sequence of positive real numbers ( s 1 , s 2 , . . . ) {\displaystyle (s_{1},s_{2},...)} is called superincreasing if every element of the sequence is greater than the sum of all previous elements in the sequence. Formally, this condition can be written as s n + 1 > ∑ j = 1 n s j {\displaystyle s_{n+1}>\sum _{j=1}^{n}s_{j}} for all n ≥ 1. == Program == The following Python source code tests a sequence of numbers to determine if it is superincreasing: This produces the following output: Sum: 0 Element: 1 Sum: 1 Element: 3 Sum: 4 Element: 6 Sum: 10 Element: 13 Sum: 23 Element: 27 Sum: 50 Element: 52 Is it a superincreasing sequence? True == Examples == (1, 3, 6, 13, 27, 52) is a superincreasing sequence, but (1, 3, 4, 9, 15, 25) is not. The series a^x for a>=2 == Properties == Multiplying a superincreasing sequence by a positive real constant keeps it superincreasing.

  Read more →
• 

  Model-based clustering

  In statistics, cluster analysis is the algorithmic grouping of objects into homogeneous groups based on numerical measurements. Model-based clustering based on a statistical model for the data, usually a mixture model. This has several advantages, including a principled statistical basis for clustering, and ways to choose the number of clusters, to choose the best clustering model, to assess the uncertainty of the clustering, and to identify outliers that do not belong to any group. == Model-based clustering == Suppose that for each of n {\displaystyle n} observations we have data on d {\displaystyle d} variables, denoted by y i = ( y i , 1 , … , y i , d ) {\displaystyle y_{i}=(y_{i,1},\ldots ,y_{i,d})} for observation i {\displaystyle i} . Then model-based clustering expresses the probability density function of y i {\displaystyle y_{i}} as a finite mixture, or weighted average of G {\displaystyle G} component probability density functions: p ( y i ) = ∑ g = 1 G τ g f g ( y i ∣ θ g ) , {\displaystyle p(y_{i})=\sum _{g=1}^{G}\tau _{g}f_{g}(y_{i}\mid \theta _{g}),} where f g {\displaystyle f_{g}} is a probability density function with parameter θ g {\displaystyle \theta _{g}} , τ g {\displaystyle \tau _{g}} is the corresponding mixture probability where ∑ g = 1 G τ g = 1 {\displaystyle \sum _{g=1}^{G}\tau _{g}=1} . Then in its simplest form, model-based clustering views each component of the mixture model as a cluster, estimates the model parameters, and assigns each observation to cluster corresponding to its most likely mixture component. === Gaussian mixture model === The most common model for continuous data is that f g {\displaystyle f_{g}} is a multivariate normal distribution with mean vector μ g {\displaystyle \mu _{g}} and covariance matrix Σ g {\displaystyle \Sigma _{g}} , so that θ g = ( μ g , Σ g ) {\displaystyle \theta _{g}=(\mu _{g},\Sigma _{g})} . This defines a Gaussian mixture model. The parameters of the model, τ g {\displaystyle \tau _{g}} and θ g {\displaystyle \theta _{g}} for g = 1 , … , G {\displaystyle g=1,\ldots ,G} , are typically estimated by maximum likelihood estimation using the expectation-maximization algorithm (EM); see also EM algorithm and GMM model. Bayesian inference is also often used for inference about finite mixture models. The Bayesian approach also allows for the case where the number of components, G {\displaystyle G} , is infinite, using a Dirichlet process prior, yielding a Dirichlet process mixture model for clustering. === Choosing the number of clusters === An advantage of model-based clustering is that it provides statistically principled ways to choose the number of clusters. Each different choice of the number of groups G {\displaystyle G} corresponds to a different mixture model. Then standard statistical model selection criteria such as the Bayesian information criterion (BIC) can be used to choose G {\displaystyle G} . The integrated completed likelihood (ICL) is a different criterion designed to choose the number of clusters rather than the number of mixture components in the model; these will often be different if highly non-Gaussian clusters are present. === Parsimonious Gaussian mixture model === For data with high dimension, d {\displaystyle d} , using a full covariance matrix for each mixture component requires estimation of many parameters, which can result in a loss of precision, generalizabity and interpretability. Thus it is common to use more parsimonious component covariance matrices exploiting their geometric interpretation. Gaussian clusters are ellipsoidal, with their volume, shape and orientation determined by the covariance matrix. Consider the eigendecomposition of a matrix Σ g = λ g D g A g D g T , {\displaystyle \Sigma _{g}=\lambda _{g}D_{g}A_{g}D_{g}^{T},} where D g {\displaystyle D_{g}} is the matrix of eigenvectors of Σ g {\displaystyle \Sigma _{g}} , A g = diag { A 1 , g , … , A d , g } {\displaystyle A_{g}={\mbox{diag}}\{A_{1,g},\ldots ,A_{d,g}\}} is a diagonal matrix whose elements are proportional to the eigenvalues of Σ g {\displaystyle \Sigma _{g}} in descending order, and λ g {\displaystyle \lambda _{g}} is the associated constant of proportionality. Then λ g {\displaystyle \lambda _{g}} controls the volume of the ellipsoid, A g {\displaystyle A_{g}} its shape, and D g {\displaystyle D_{g}} its orientation. Each of the volume, shape and orientation of the clusters can be constrained to be equal (E) or allowed to vary (V); the orientation can also be spherical, with identical eigenvalues (I). This yields 14 possible clustering models, shown in this table: It can be seen that many of these models are more parsimonious, with far fewer parameters than the unconstrained model that has 90 parameters when G = 4 {\displaystyle G=4} and d = 9 {\displaystyle d=9} . Several of these models correspond to well-known heuristic clustering methods. For example, k-means clustering is equivalent to estimation of the EII clustering model using the classification EM algorithm. The Bayesian information criterion (BIC) can be used to choose the best clustering model as well as the number of clusters. It can also be used as the basis for a method to choose the variables in the clustering model, eliminating variables that are not useful for clustering. Different Gaussian model-based clustering methods have been developed with an eye to handling high-dimensional data. These include the pgmm method, which is based on the mixture of factor analyzers model, and the HDclassif method, based on the idea of subspace clustering. The mixture-of-experts framework extends model-based clustering to include covariates. == Example == We illustrate the method with a dateset consisting of three measurements (glucose, insulin, sspg) on 145 subjects for the purpose of diagnosing diabetes and the type of diabetes present. The subjects were clinically classified into three groups: normal, chemical diabetes and overt diabetes, but we use this information only for evaluating clustering methods, not for classifying subjects. The BIC plot shows the BIC values for each combination of the number of clusters, G {\displaystyle G} , and the clustering model from the Table. Each curve corresponds to a different clustering model. The BIC favors 3 groups, which corresponds to the clinical assessment. It also favors the unconstrained covariance model, VVV. This fits the data well, because the normal patients have low values of both sspg and insulin, while the distributions of the chemical and overt diabetes groups are elongated, but in different directions. Thus the volumes, shapes and orientations of the three groups are clearly different, and so the unconstrained model is appropriate, as selected by the model-based clustering method. The classification plot shows the classification of the subjects by model-based clustering. The classification was quite accurate, with a 12% error rate as defined by the clinical classification. Other well-known clustering methods performed worse with higher error rates, such as single-linkage clustering with 46%, average link clustering with 30%, complete-linkage clustering also with 30%, and k-means clustering with 28%. == Outliers in clustering == An outlier in clustering is a data point that does not belong to any of the clusters. One way of modeling outliers in model-based clustering is to include an additional mixture component that is very dispersed, with for example a uniform distribution. Another approach is to replace the multivariate normal densities by t {\displaystyle t} -distributions, with the idea that the long tails of the t {\displaystyle t} -distribution would ensure robustness to outliers. However, this is not breakdown-robust. A third approach is the "tclust" or data trimming approach which excludes observations identified as outliers when estimating the model parameters. == Non-Gaussian clusters and merging == Sometimes one or more clusters deviate strongly from the Gaussian assumption. If a Gaussian mixture is fitted to such data, a strongly non-Gaussian cluster will often be represented by several mixture components rather than a single one. In that case, cluster merging can be used to find a better clustering. A different approach is to use mixtures of complex component densities to represent non-Gaussian clusters. == Non-continuous data == === Categorical data === Clustering multivariate categorical data is most often done using the latent class model. This assumes that the data arise from a finite mixture model, where within each cluster the variables are independent. === Mixed data === These arise when variables are of different types, such as continuous, categorical or ordinal data. A latent class model for mixed data assumes local independence between the variable. The location model relaxes the local independence assumption. The clustMD approach assumes that the observed variables are manifestations of underlying continuous Gaussian latent

  Read more →
• 

  SIPRNet

  The Secret Internet Protocol Router Network (SIPRNet) is "a system of interconnected computer networks used by the U.S. Department of Defense and the U.S. Department of State to transmit classified information (up to and including information classified SECRET) by packet switching over the 'completely secure' environment". It also provides services such as hypertext document access and electronic mail. SIPRNet is a component of the Defense Information Systems Network. Other components handle communications with other security needs, such as the NIPRNet, which is used for nonsecure communications, and the Joint Worldwide Intelligence Communications System (JWICS), which is used for Top Secret communications. == Access == According to the U.S. Department of State Web Development Handbook, domain structure and naming conventions are the same as for the open internet, except for the addition of a second-level domain, like, e.g., "sgov" between state and gov: openforum.state.sgov.gov. Files originating from SIPRNet are marked by a header tag "SIPDIS" (SIPrnet DIStribution). A corresponding second-level domain smil.mil exists for DoD users. Access is also available to a "...small pool of trusted allies, including Australia, Canada, the United Kingdom and New Zealand...". This group (including the US) is known as the Five Eyes. SIPRNet was one of the networks accessed by Chelsea Manning, convicted of leaking the video used in WikiLeaks' "Collateral Murder" release as well as the source of the US diplomatic cables published by WikiLeaks in November 2010. == Alternate names == SIPRNet and NIPRNet are referred to colloquially as SIPPERnet and NIPPERnet (or simply sipper and nipper), respectively.

  Read more →
• 

  Content management

  Content management (CM) are a set of processes and technologies that support the collection, managing, and publishing of information in any form or medium. When stored and accessed via computers, this information may be more specifically referred to as digital content, or simply as content. Digital content may take the form of text (such as electronic documents), images, multimedia files (such as audio or video files), or any other file type that follows a content lifecycle requiring management. The process of content development and management is complex enough that various commercial software vendors (large and small), such as Interwoven and Microsoft, offer content management software to control and automate significant aspects of the content lifecycle. == Process == Content management practices and goals vary by mission and by organizational governance structure. News organizations, e-commerce websites, and educational institutions all use content management, but in different ways. This leads to differences in terminology and in the names and number of steps in the process. For example, some digital content is created by one or more authors. Over time that content may be edited. One or more individuals may provide some editorial oversight, approving the content for publication. Publishing may take many forms: it may be the act of "pushing" content out to others, or simply granting digital access rights to certain content to one or more individuals. Later that content may be superseded by another version of the content and thus retired or removed from use (as when this wiki page is modified). Content management is an inherently collaborative process. It often consists of the following basic roles and responsibilities: Creator – responsible for creating and editing content. Editor – responsible for tuning the content message and the style of delivery, including translation and localization. Publisher – responsible for releasing the content for use. Administrator – responsible for managing access permissions to folders, collections and files, usually accomplished by assigning access rights to user groups or roles. Admins may also assist and support users in various ways. Consumer, viewer or guest – the person who reads or otherwise consumes the content after it is published or shared. A critical aspect of content management is the ability to manage versions of content as it evolves (see also version control). Authors and editors often need to restore older versions of edited products due to a process failure or an undesirable series of edits. Time-sensitive content may also require updates as the subject matter evolves over time. Another equally important aspect of content management involves the creation, maintenance, and application of review standards. Each member of the content creation and review process has a unique role and set of responsibilities in the development or publication of the content. Each review team member requires clear and concise review standards. These must be maintained on an ongoing basis to ensure the long-term consistency and health of the knowledge base. A content management system is a set of automated processes that may support the following features: Import and creation of documents and multimedia material Identification of all key users and their roles The ability to assign roles and responsibilities to different instances of content categories or types Definition of workflow tasks often coupled with messaging so that content managers are alerted to changes in content The ability to track and manage multiple versions of a single instance of content The ability to publish the content to a repository to support access The ability to personalize content based on a set of rules Increasingly, the repository is an inherent part of the system, and incorporates enterprise search and retrieval. Content management systems take the following forms: Web content management system—software for web site management (often what content management implicitly means) Output of a newspaper editorial staff organization Workflow for article publication Document management systems Knowledge management software Single source content management system—content stored in chunks within a relational database Variant management system—where personnel tag source content (usually text and graphics) to represent variants stored as single source "master" content modules, resolved to the desired variant at publication (for example: automobile owners manual content for 12 model years stored as single master content files and "called" by model year as needed)—often used in concert with database chunk storage (see above) for large content objects == Governance structures == Content management expert Marc Feldman defines three primary content management governance structures: localized, centralized, and federated—each having its unique strengths and weaknesses. === Localized governance === By putting control in the hands of those closest to the content, the context experts, localized governance models empower and unleash creativity. These benefits come, however, at the cost of a partial-to-total loss of managerial control and oversight. === Centralized governance === When the levers of control are strongly centralized, content management systems are capable of delivering an exceptionally clear and unified brand message. Moreover, centralized content management governance structures allow for a large number of cost-savings opportunities in large enterprises, realized, for example, through (1) the avoidance of duplicated efforts in creating, editing, formatting, repurposing and archiving content; (2) process management and the streamlining of all content related labor; and/or (3) an orderly deployment or updating of the content management system. === Federated governance === Federated governance models potentially realize the benefits of both localized and centralized control while avoiding the weaknesses of both. While content management software systems are inherently structured to enable federated governance models, realizing these benefits can be difficult because it requires, for example, negotiating the boundaries of control with local managers and content creators. In the case of larger enterprises, in particular, the failure to fully implement or realize a federated governance structure equates to a failure to realize the full return on investment and cost savings that content management systems enable. == Implementation == Content management implementations must be able to manage content distributions and digital rights in content life cycle. Content management systems are usually involved with digital rights management in order to control user access and digital rights. In this step, the read-only structures of digital rights management systems force some limitations on content management, as they do not allow authors to change protected content in their life cycle. Creating new content using managed (protected) content is also an issue that gets protected contents out of management controlling systems. A few content management implementations cover all these issues.

  Read more →
• 

  Plaintext

  In cryptography, plaintext usually means unencrypted information pending input into cryptographic algorithms, usually encryption algorithms. This usually refers to data that is transmitted or stored unencrypted. == Overview == With the advent of computing, the term plaintext expanded beyond human-readable documents to mean any data, including binary files, in a form that can be viewed or used without requiring a key or other decryption device. Information—a message, document, file, etc.—if to be communicated or stored in an unencrypted form is referred to as plaintext. Plaintext is used as input to an encryption algorithm; the output is usually termed ciphertext, particularly when the algorithm is a cipher. Codetext is less often used, and almost always only when the algorithm involved is actually a code. Some systems use multiple layers of encryption, with the output of one encryption algorithm becoming "plaintext" input for the next. == Secure handling == Insecure handling of plaintext can introduce weaknesses into a cryptosystem by letting an attacker bypass the cryptography altogether. Plaintext is vulnerable in use and in storage, whether in electronic or paper format. Physical security means the securing of information and its storage media from physical, attack—for instance by someone entering a building to access papers, storage media, or computers. Discarded material, if not disposed of securely, may be a security risk. Even shredded documents and erased magnetic media might be reconstructed with sufficient effort. If plaintext is stored in a computer file, the storage media, the computer and its components, and all backups must be secure. Sensitive data is sometimes processed on computers whose mass storage is removable, in which case physical security of the removed disk is vital. In the case of securing a computer, useful (as opposed to handwaving) security must be physical (e.g., against burglary, brazen removal under cover of supposed repair, installation of covert monitoring devices, etc.), as well as virtual (e.g., operating system modification, illicit network access, Trojan programs). Wide availability of keydrives, which can plug into most modern computers and store large quantities of data, poses another severe security headache. A spy (perhaps posing as a cleaning person) could easily conceal one, and even swallow it if necessary. Discarded computers, disk drives and media are also a potential source of plaintexts. Most operating systems do not actually erase anything— they simply mark the disk space occupied by a deleted file as 'available for use', and remove its entry from the file system directory. The information in a file deleted in this way remains fully present until overwritten at some later time when the operating system reuses the disk space. With even low-end computers commonly sold with many gigabytes of disk space and rising monthly, this 'later time' may be months later, or never. Even overwriting the portion of a disk surface occupied by a deleted file is insufficient in many cases. Peter Gutmann of the University of Auckland wrote a celebrated 1996 paper on the recovery of overwritten information from magnetic disks; areal storage densities have gotten much higher since then, so this sort of recovery is likely to be more difficult than it was when Gutmann wrote. Modern hard drives automatically remap failing sectors, moving data to good sectors. This process makes information on those failing, excluded sectors invisible to the file system and normal applications. Special software, however, can still extract information from them. Some government agencies (e.g., US NSA) require that personnel physically pulverize discarded disk drives and, in some cases, treat them with chemical corrosives. This practice is not widespread outside government, however. Garfinkel and Shelat (2003) analyzed 158 second-hand hard drives they acquired at garage sales and the like, and found that less than 10% had been sufficiently sanitized. The others contained a wide variety of readable personal and confidential information. See data remanence. Physical loss is a serious problem. The US State Department, Department of Defense, and the British Secret Service have all had laptops with secret information, including in plaintext, lost or stolen. Appropriate disk encryption techniques can safeguard data on misappropriated computers or media. On occasion, even when data on host systems is encrypted, media that personnel use to transfer data between systems is plaintext because of poorly designed data policy. For example, in October 2007, HM Revenue and Customs lost CDs that contained the unencrypted records of 25 million child benefit recipients in the United Kingdom. Modern cryptographic systems resist known plaintext or even chosen plaintext attacks, and so may not be entirely compromised when plaintext is lost or stolen. Older systems resisted the effects of plaintext data loss on security with less effective techniques—such as padding and Russian copulation to obscure information in plaintext that could be easily guessed.

  Read more →
• 

  Language identification

  In natural language processing, language identification or language guessing is the problem of determining which natural language a given content is in. Computational approaches to this problem view it as a special case of text categorization, solved with various statistical methods. == Overview == === Logical approach === A common non-statistical intuitive approach (though highly uncertain) is to look for common letter combinations, or distinctive diacritics or punctuation. === Statistical approach === There are several statistical approaches to language identification. An older statistical method by Grefenstette was based on the frequency of short n-grams, which are often function morphemes. For example, "ing" is more common in English than in French, while the sequence "que" is more common in French. Given a new page found on the Web, one counts the number of occurrences of each such short sequence and picks the language whose frequency table it matches the most. One technique is to compare the compressibility of the text to the compressibility of texts in a set of known languages. This approach is known as mutual information based distance measure. The same technique can also be used to empirically construct family trees of languages which closely correspond to the trees constructed using historical methods. Mutual information based distance measure is essentially equivalent to more conventional model-based methods and is not generally considered to be either novel or better than simpler techniques. Another technique, as described by Cavnar and Trenkle (1994) and Dunning (1994) is to create a language n-gram model from a "training text" for each of the languages. These models can be based on characters (Cavnar and Trenkle) or encoded bytes (Dunning); in the latter, language identification and character encoding detection are integrated. Then, for any piece of text needing to be identified, a similar model is made, and that model is compared to each stored language model. The most likely language is the one with the model that is most similar to the model from the text needing to be identified. This approach can be problematic when the input text is in a language for which there is no model. In that case, the method may return another, "most similar" language as its result. Also problematic for any approach are pieces of input text that are composed of several languages, as is common on the Web. As of 2025, a commonly used baseline method is via the fastText library, which has comparable classification accuracy as deep learning techniques, but much faster. == Identifying similar languages == One of the great bottlenecks of language identification systems is to distinguish between closely related languages. Similar languages like Bulgarian and Macedonian or Indonesian and Malay present significant lexical and structural overlap, making it challenging for systems to discriminate between them. In 2014 the DSL shared task has been organized providing a dataset (Tan et al., 2014) containing 13 different languages (and language varieties) in six language groups: Group A (Bosnian, Croatian, Serbian), Group B (Indonesian, Malaysian), Group C (Czech, Slovak), Group D (Brazilian Portuguese, European Portuguese), Group E (Peninsular Spanish, Argentine Spanish), Group F (American English, British English). The best system reached performance of over 95% results (Goutte et al., 2014). Results of the DSL shared task are described in Zampieri et al. 2014. == Software == Apache OpenNLP includes char n-gram based statistical detector and comes with a model that can distinguish 103 languages Apache Tika contains a language detector for 18 languages

  Read more →
• 

  Public Services Network

  The Public Services Network (PSN) is a UK government's high-performance network, which helps public sector organisations work together, reduce duplication and share resources. It unified the provision of network infrastructure across the United Kingdom public sector into an interconnected "network of networks" to increase efficiency and reduce overall public expenditure. It is now a legacy network and public sector organisations are being migrated to using services on the public internet. == Origins == The Public Services Network (PSN) was launched officially as part of the Transformational Government Strategy commencing in 2005, under the original name of the Public Sector Network. Prior to this, some parts of local government had already successfully implemented the concept. The Hampshire Public Services Network (HPSN) was the first PSN, launched in 1999, followed closely by Kent County Councils partnerships with the KPSN. The HPSN, encompassing all of the borough, district and unitary councils, with the County Council, as well as the Fire Services, the Isle of Wight Council and 540 schools. National PSN technical and architecture compliance criteria were established from 2007, by GDS working with local government leaders from Socitm (the Society of Information Technology Management) on the National CIO Council and the Local CIO Council. The PSN's aim was to bring public services organisations with a common interest onto a single, coherent and standards-based ‘network of networks’. This would create influence, economies of scale and a commonality of standards for secure and easy inter-connection between public service organisations. The original concept of a network of networks strategy was based upon the work already undertaken in local government and recognition of Communities of Interest (COI) within the Criminal Justice Sector during work by the Office for Criminal Justice Reform (OCJR) between 2005 and 2007 to enable data sharing across business units. In this context a COI was defined as groups of Government departments and external partners who in combination provided services within a specific area of operation and used the same data, with a similar risk profile, shared risk appetite and common governance framework. Historically each group member had implemented their own networks and standards of operation in isolation with little or no consideration as to how services and data may be shared and resulting in increased costs of operation. The Network of Networks strategy proposed within OCJR recommended the creation of specific networks based upon these Communities of Interest which were joined together through data interchange gateways supporting common standards. Under this approach networks would be arranged by data type and business functions such as Criminal Justice, Health and Social Care, Defence and Intelligence or Public Finance rather than solely on established departmental boundaries. Within a COI, trust relationships and data interchange are readily supported, enabling data sharing without a need to cross network boundaries and providing benefits of scale without the challenges and compromises intrinsic to homogeneous cross sector networks. Data is made available without a need to transport it between organisations and control is retained by the data originator. In early 2007 a group of UK Government department CTOs in conjunction with the Office for Government Commerce Buying Solutions (OGC BS) established the vision for a single commonly provided, procured and managed public sector voice and data network infrastructure to replace the multitude of separately procured and managed networks serving various segments of the UK public sector; Education, Health, Central Government, Local Government etc. In 2008 an Industry Working Group was established to document the objectives and requirements more clearly. Their report set out the architectural and commercial principles as well as anticipated security, service management, governance and transition arrangements. == Architecture == The PSN comprises a core network, the Government Conveyancing Network or GCN provided by GCN Service Providers or GCNSPs. The GCN interconnects multiple operator networks, termed Direct Network Service Providers or DNSPs. Subscriber organisations contract to a connection from a local participating DNSP, connect via that to GCN and hence onwards to other interconnected networks and services. The GCN network is entirely based on IPv4 and MPLS and the GCNSPs are not currently mandated to provide IPv6, though they should have a roadmap to implementing it if and when required. == Commercial framework == In 2010 Virgin Media Business, BT, Cable & Wireless and Global Crossing signed Deeds of Undertaking (DoU) and subsequently achieved accreditation for providing GCN and IP VPN services. In March 2012, BT, Cable & Wireless, Capita Business Services, Eircom, Fujitsu, Kcom, Level 3, Logicalis, MDNX, Thales, Updata and Virgin Media Business were successful bidders for the initial two-year PSN Connectivity framework. In June 2012, 29 companies were confirmed as suppliers of ICT services to the UK public sector under the Government's PSN Services framework contract. Apart from most of the previous suppliers, additional companies also included 2e2, Airwave Solutions, Azzurri Communications, Cassidian, CSC Computer Sciences, Computacenter, Daisy Communications, Easynet Global Services, EE, Freedom Communications, Icom Holdings, NextiraOne, PageOne Communications, Phoenix IT Group, Siemens Communications, Specialist Computer Centres, Telefónica, telent Technology Services, Uniworld Communications and Vodafone. == Governance == The PSN is managed within the Cabinet Office where it is part of the Government Digital Service. == Early implementations == There were already notable initiatives in progress in county council areas, demonstrating public sector network integration in both the Hampshire HPSN2 network and in Kent's community network. Project Pathway was established as a pilot linking these two county-wide networks, with Virgin Media Business and Global Crossing the subscriber and GCN network elements. Staffordshire County Council was the first council in England to establish a PSN that included the county's NHS Health partners. Other county councils have since followed the leads of these councils. == Transition == Centrally procured public sector networks are expected to migrate across to the PSN framework as they reach the end of their contract terms, either through an interim framework or directly. The Government Secure Intranet (GSi) contracts expired in September 2011, running on to 12 February 2012 and were replaced by the transitional Government Secure Intranet Convergence Framework (GCF). The Managed Telephony Service (MTS) contract expired on 31 December 2011 and was replaced by the Managed Telephony Convergence Framework (MTCF). == Future plan == In a blog post published on 20 January 2017, Government Digital Service announced that the Technology Leaders Network (TLN) had agreed that government was starting a journey away from the PSN. This was because using the Internet was considered suitable for the vast majority of the work that the public sector does. The blog post confirmed that the 'move was not going to happen immediately' and stated that 'there's quite a bit of work to do across the public sector to prepare for the changes'. It also stated that it was too early for a full timeline to be provided, although all PSN-connected organisations would be updated as the process evolved. The blog post confirmed that organisations that need to access services that are only available on the PSN would still need to connect to it for the time being and continue to meet its assurance requirements. In a blog post published on 16 March 2017, Government Digital Service (GDS) set out its plans for PSN assurance. The blog post confirmed that the PSN compliance process wasn't 'going anywhere, certainly for a while yet'. It explained that the TLN agreed that – as one of the only recognised, externally accredited, cross-government common assurance standards – it 'needs to live on far beyond the end of the physical PSN network'. Government Digital Service, along with the National Cyber Security Centre (NCSC) and the Cyber and Government Security Directorate, are now looking at ways to expand and reframe PSN compliance in a new context that, while retaining the assurance principles that are the basis of the existing process, will aim to improve the process. A GDS blog post titled 'The road to closing down the PSN' published on 8 September 2020 describes how the public sector will migrate away from the PSN. The Cabinet Office has set up a programme called Future Networks for Government (FN4G) to help organisations move away from the PSN.

  Read more →
• 

  Frame (networking)

  A frame is a digital data transmission unit in computer networking and telecommunications. In packet switched systems, a frame is a simple container for a single network packet. In other telecommunications systems, a frame is a repeating structure supporting time-division multiplexing. A frame typically includes frame synchronization features consisting of a sequence of bits or symbols that indicate to the receiver the beginning and end of the payload data within the stream of symbols or bits it receives. If a receiver is connected to the system during frame transmission, it ignores the data until it detects a new frame synchronization sequence. == Packet switching == In the OSI model of computer networking, a frame is the protocol data unit at the data link layer. Frames are the result of the final layer of encapsulation before the data is transmitted over the physical layer. A frame is "the unit of transmission in a link layer protocol, and consists of a link layer header followed by a packet." Each frame is separated from the next by an interframe gap. A frame is a series of bits generally composed of frame synchronization bits, the packet payload, and a frame check sequence. Examples are Ethernet frames, Wi-Fi frames, 4G frames, Point-to-Point Protocol (PPP) frames, Fibre Channel frames, and V.42 modem frames. Often, frames of several different sizes are nested inside each other. For example, when using Point-to-Point Protocol (PPP) over asynchronous serial communication, the eight bits of each individual byte are framed by start and stop bits, the payload data bytes in a network packet are framed by the header and footer, and several packets can be framed with frame boundary octets. == Time-division multiplex == In telecommunications, specifically in time-division multiplex (TDM) and time-division multiple access (TDMA) variants, a frame is a cyclically repeated data block that consists of a fixed number of time slots, one for each logical TDM channel or TDMA transmitter. In this context, a frame is typically an entity at the physical layer. TDM application examples are SONET/SDH and the ISDN circuit-switched B-channel, while TDMA examples are Circuit Switched Data used in early cellular voice services. The frame is also an entity for time-division duplex, where the mobile terminal may transmit during some time slots and receive during others.

  Read more →
• 

  POODLE

  POODLE (which stands for "Padding Oracle On Downgraded Legacy Encryption") is a security vulnerability which takes advantage of the fallback to SSL 3.0. If attackers successfully exploit this vulnerability, on average, they only need to make 256 SSL 3.0 requests to reveal one byte of encrypted messages. Bodo Möller, Thai Duong and Krzysztof Kotowicz from the Google Security Team discovered this vulnerability; they disclosed the vulnerability publicly on October 14, 2014 (despite the paper being dated "September 2014"). On December 8, 2014, a variation of the POODLE vulnerability that affected TLS was announced. The CVE-ID associated with the original POODLE attack is CVE-2014-3566. F5 Networks filed for CVE-2014-8730 as well, see POODLE attack against TLS section below. == Prevention == To mitigate the POODLE attack, one approach is to completely disable SSL 3.0 on the client side and the server side. However, some old clients and servers do not support TLS 1.0 and above. Thus, the authors of the paper on POODLE attacks also encourage browser and server implementation of TLS_FALLBACK_SCSV, which will make downgrade attacks impossible. Another mitigation is to implement "anti-POODLE record splitting". It splits the records into several parts and ensures none of them can be attacked. However the problem of the splitting is that, though valid according to the specification, it may also cause compatibility issues due to problems in server-side implementations. A full list of browser versions and levels of vulnerability to different attacks (including POODLE) can be found in the article Transport Layer Security. Opera 25 implemented this mitigation in addition to TLS_FALLBACK_SCSV. Google's Chrome browser and their servers had already supported TLS_FALLBACK_SCSV. Google stated in October 2014 it was planning to remove SSL 3.0 support from their products completely within a few months. Fallback to SSL 3.0 has been disabled in Chrome 39, released in November 2014. SSL 3.0 has been disabled by default in Chrome 40, released in January 2015. Mozilla disabled SSL 3.0 in Firefox 34 and ESR 31.3, which were released in December 2014, and added support of TLS_FALLBACK_SCSV in Firefox 35. Microsoft published a security advisory to explain how to disable SSL 3.0 in Internet Explorer and Windows OS, and on October 29, 2014, Microsoft released a fix which disables SSL 3.0 in Internet Explorer on Windows Vista / Server 2003 and above and announced a plan to disable SSL 3.0 by default in their products and services within a few months. Microsoft disabled fallback to SSL 3.0 in Internet Explorer 11 for Protect Mode sites on February 10, 2015, and for other sites on April 14, 2015. Apple's Safari (on OS X 10.8, iOS 8.1 and later) mitigated against POODLE by removing support for all CBC protocols in SSL 3.0, however, this left RC4 which is also completely broken by the RC4 attacks in SSL 3.0. POODLE was completely mitigated in OS X 10.11 (El Capitan 2015) and iOS 9 (2015). To prevent the POODLE attack, some web services dropped support of SSL 3.0. Examples include CloudFlare and Wikimedia. Network Security Services version 3.17.1 (released on October 3, 2014) and 3.16.2.3 (released on October 27, 2014) introduced support for TLS_FALLBACK_SCSV, and NSS will disable SSL 3.0 by default in April 2015. OpenSSL versions 1.0.1j, 1.0.0o and 0.9.8zc, released on October 15, 2014, introduced support for TLS_FALLBACK_SCSV. LibreSSL version 2.1.1, released on October 16, 2014, disabled SSL 3.0 by default. == POODLE attack against TLS == A new variant of the original POODLE attack was announced on December 8, 2014. This attack exploits implementation flaws of CBC encryption mode in the TLS 1.0 - 1.2 protocols. Even though TLS specifications require servers to check the padding, some implementations fail to validate it properly, which makes some servers vulnerable to POODLE even if they disable SSL 3.0. SSL Pulse showed "about 10% of the servers are vulnerable to the POODLE attack against TLS" before this vulnerability was announced. The CVE-ID for F5 Networks' implementation bug is CVE-2014-8730. The entry in NIST's NVD states that this CVE-ID is to be used only for F5 Networks' implementation of TLS, and that other vendors whose products have the same failure to validate the padding mistake in their implementations like A10 Networks and Cisco Systems need to issue their own CVE-IDs for their implementation errors because this is not a flaw in the protocol but in the implementation. The POODLE attack against TLS was found to be easier to initiate than the initial POODLE attack against SSL. There is no need to downgrade clients to SSL 3.0, meaning fewer steps are needed to execute a successful attack.

  Read more →
• 

  Amazon Elastic Compute Cloud

  Amazon Elastic Compute Cloud (EC2) is a part of Amazon's cloud-computing platform, Amazon Web Services (AWS), that allows users to rent virtual computers on which to run their own computer applications. EC2 encourages scalable deployment of applications by providing a web service through which a user can boot an Amazon Machine Image (AMI) to configure a virtual machine, which Amazon calls an "instance", containing any software desired. A user can create, launch, and terminate server-instances as needed, paying by the second for active servers – hence the term "elastic". EC2 provides users with control over the geographical location of instances that allows for latency optimization and high levels of redundancy. In November 2010, Amazon switched its own retail website platform to EC2 and AWS. == History == Amazon announced a limited public beta test of EC2 on August 25, 2006, offering access on a first-come, first-served basis. Amazon added two new instance types (Large and Extra-Large) on October 16, 2007. On May 29, 2008, two more types were added, High-CPU Medium and High-CPU Extra Large. There were twelve types of instances available. Amazon added three new features on March 27, 2008: static IP addresses, availability zones, and user-selectable kernels. On August 20, 2008, Amazon added Elastic Block Store (EBS). This provides persistent storage, a feature that had been lacking since the service was introduced. Amazon EC2 went into full production when it dropped the beta label on October 23, 2008. On the same day, Amazon announced the following features: a service level agreement for EC2, Microsoft Windows in beta form on EC2, Microsoft SQL Server in beta form on EC2, plans for an AWS management console, and plans for load balancing, autoscaling, and cloud monitoring services. These features were subsequently added on May 18, 2009. Amazon EC2 was developed mostly by a team in Cape Town, South Africa led by Chris Pinkham. Pinkham provided the initial architecture guidance for EC2 and then built the team and led the development of the project along with Willem van Biljon. == Instance types == Initially, EC2 used Xen virtualization exclusively. However, on November 6, 2017, Amazon announced the new C5 family of instances that were based on a custom architecture around the KVM hypervisor, called Nitro. Each virtual machine, called an "instance", functions as a virtual private server. Amazon sizes instances based on "Elastic Compute Units". The performance of otherwise identical virtual machines may vary. On November 28, 2017, AWS announced a bare-metal instance, a departure from exclusively offering virtualized instance types. As of January 2019, the following instance types were offered: General Purpose: A1, T3, T2, M5, M5a, M4, T3a Compute Optimized: C5, C5n, C4 Memory Optimized: R5, R5a, R4, X1e, X1, High Memory, z1d Accelerated Computing: P3, P2, G3, F1 Storage Optimized: H1, I3, D2 As of April 2018, the following payment methods by instance were offered: On-demand: pay by the hour without commitment. Reserved: rent instances with one-time payment receiving discounts on the hourly charge. Spot: bid-based service: runs the jobs only if the spot price is below the bid specified by bidder. The spot price is claimed to be supply-demand based, however a 2011 study concluded that the price was generally not set to clear the market, but was dominated by an undisclosed reserve price. In 2025, AWS expanded EC2 with the compute-optimized C8gn family, powered by Graviton4 and offering up to 600 Gbit/s network bandwidth (about 30% higher compute performance than C7gn), and introduced G6f fractional-GPU instances that let customers provision one-eighth, one-quarter, or one-half of an NVIDIA L4 GPU for right-sized graphics/ML workloads. === Cost === As of April 2018, Amazon charged about $0.0058 per hour ($4.176 per month) for the smallest "Nano Instance" (t2.nano) virtual machine running Linux or Windows. Storage-optimized instances cost as much as $4.992 per hour (i3.16xlarge). "Reserved" instances can go as low as $2.50 per month for a three-year prepaid plan. The data transfer charge ranges from free to $0.12 per gigabyte, depending on the direction and monthly volume (inbound data transfer is free on all AWS services). EC2 costs can be analyzed using the Amazon Cost and Usage Report. There are many different cost categories for EC2 including: hourly Instance Charges, Data Transfer, EBS Volumes, EBS Volume Snapshots, and Nat Gateway. === Free tier === As of December 2010 Amazon offered a bundle of free resource credits to new account holders. The credits are designed to run a "micro" sized server, storage (EBS), and bandwidth for one year. Unused credits cannot be carried over from one month to the next. === Reserved instances === Reserved instances enable EC2 or RDS service users to reserve an instance for one or three years. The corresponding hourly rate charged by Amazon to operate the instance is 35 to 75% lower than the rate charged for on-demand instances. Reserved instances can be purchased with three different payment options: All Upfront, Partial Upfront and No Upfront. The different purchase options allow for different structuring of payment models, with a larger discount given to customers that pay their reservation upfront. Reserved Instances are purchased based on a resource commitment. These reservations are made based on an instance type and a count of that instance type. For example, you could reserve 100 i3.large instances for a 3-year term. In September 2016, AWS announced several enhancements to Reserved instances, introducing a new feature called scope and a new reservation type called a Convertible. In October 2017, AWS announced the allowance to subdivide the instances purchased for more flexibility. === Spot instances === Cloud providers maintain large amounts of excess capacity they have to sell or risk incurring losses. Amazon EC2 Spot instances are spare compute capacity in the AWS cloud available at up to 90% discount compared to On-Demand prices. As a trade-off, AWS offers no SLA on these instances and customers take the risk that it can be interrupted with only two minutes of notification when Amazon needs the capacity back. Researchers from the Israeli Institute of Technology found that "they (Spot instances) are typically generated at random from within a tight price interval via a dynamic hidden reserve price". Some companies, like Spotinst, are using machine learning to predict spot interruptions up to 15 minutes in advance. === Savings Plans === In November 2019, Amazon announced Savings Plans. Savings Plans are an alternative to Reserved Instances that come in two different plan types: Compute Savings Plans and EC2 Instances Savings Plans. Compute Savings Plans allow an organization to commit to EC2 and Fargate usage with the freedom to change region, family, size, availability zone, OS and tenancy inside the lifespan of the commitment. EC2 Instance Savings plans provide a larger discount than Compute Savings Plans but are less flexible meaning a user must commit to individual instance families within a region to take advantage, but with the freedom to change instances within the family in that region. AWS uses the Cost Explorer to automatically calculate recommendations for the commitments you should make how that commitment will look like as a monthly charge on your AWS bill. AWS Savings Plans are purchased based on hourly spend commitment. This hourly commitment is made using the discounted pricing of the savings plan you are purchasing. For example, you could commit to spending $5 per hour, on a Compute Savings Plan, for a 3-year term. == Features == === Operating systems === When it launched in August 2006, the EC2 service offered Linux and later Sun Microsystems' OpenSolaris and Solaris Express Community Edition. In October 2008, EC2 added the Windows Server 2003 and Windows Server 2008 operating systems to the list of available operating systems. In March 2011, NetBSD AMIs became available. In November 2012, Windows Server 2012 support was added. Since 2006, Colin Percival, a FreeBSD developer and Security Officer, solicited Amazon to add FreeBSD. In November 2012, Amazon officially supported running FreeBSD in EC2. The FreeBSD/EC2 platform is maintained by Percival who also developed the secure deduplicating Amazon S3-cloud based backup service Tarsnap. Amazon has their own Linux distribution based on Fedora and Red Hat Enterprise Linux as a low cost offering known as the Amazon Linux AMI. Version 2013.03 included: Linux kernel, Java OpenJDK Runtime Environment and GNU Compiler Collection. On November 30, 2020, Amazon announced that it would be adding macOS to the EC2 service. Initial support was announced for macOS Mojave and macOS Catalina running on Mac Mini. === Managed Container and Kubernetes Services === Amazon Elastic Container Registry (ECR) is a Docker registry service for Amazon EC2

  Read more →
• 

  Data set (IBM mainframe)

  In the context of IBM mainframe computers in the IBM System/360 line and its successors, a data set (IBM preferred) or dataset is a computer file having a record organization. Use of this term began with, e.g., DOS/360 and OS/360, and is still used by their successors, including the current VSE and z/OS. Documentation for these systems historically preferred this term rather than file. A data set is typically stored on a direct access storage device (DASD) or magnetic tape, however unit record devices, such as punch card readers, card punches, line printers and page printers can provide input/output (I/O) for a data set (file). Data sets are not unstructured streams of bytes, but rather are organized in various logical record and block structures determined by the DSORG (data set organization), RECFM (record format), and other parameters. These parameters are specified at the time of the data set allocation (creation), for example with Job Control Language DD statements. Within a running program they are stored in the Data Control Block (DCB) or Access Control Block (ACB), which are data structures used to access data sets using access methods. Records in a data set may be fixed, variable, or “undefined” length. == Data set organization == For OS/360, the DCB's DSORG parameter specifies how the data set is organized. It may be CQ Queued Telecommunications Access Method (QTAM) in Message Control Program (MCP) CX Communications line group DA Basic Direct Access Method (BDAM) GS Graphics device for Graphics Access Method(GAM) IS Indexed Sequential Access Method (ISAM) MQ QTAM message queue in application PO Partitioned Organization PS Physical Sequential among others. Data sets on tape may only be DSORG=PS. The choice of organization depends on how the data is to be accessed, and in particular, how it is to be updated. Programmers utilize various access methods (such as QSAM or VSAM) in programs for reading and writing data sets. Access method depends on the given data set organization. == Record format (RECFM) == Regardless of organization, the physical structure of each record is essentially the same, and is uniform throughout the data set. This is specified in the DCB RECFM parameter. RECFM=F means that the records are of fixed length, specified via the LRECL parameter. RECFM=V specifies a variable-length record. V records when stored on media are prefixed by a Record Descriptor Word (RDW) containing the integer length of the record in bytes and flag bits. With RECFM=FB and RECFM=VB, multiple logical records are grouped together into a single physical block on tape or DASD. FB and VB are fixed-blocked, and variable-blocked, respectively. RECFM=U (undefined) is also variable length, but the length of the record is determined by the length of the block rather than by a control field. The BLKSIZE parameter specifies the maximum length of the block. RECFM=FBS could be also specified, meaning fixed-blocked standard, meaning all the blocks except the last one were required to be in full BLKSIZE length. RECFM=VBS, or variable-blocked spanned, means a logical record could be spanned across two or more blocks, with flags in the RDW indicating whether a record segment is continued into the next block and/or was continued from the previous one. This mechanism eliminates the need for using any "delimiter" byte value to separate records. Thus data can be of any type, including binary integers, floating-point, or characters, without introducing a false end-of-record condition. The data set is an abstraction of a collection of records, in contrast to files as unstructured streams of bytes. == Partitioned data set == A partitioned data set (PDS) is a data set containing multiple members, each of which holds a separate sub-data set, similar to a directory in other types of file systems. This type of data set is often used to hold load modules (old format bound executable programs), source program libraries (especially Assembler macro definitions), ISPF screen definitions, and Job Control Language. A PDS may be compared to a Zip file or COM Structured Storage. A Partitioned Data Set can only be allocated on a single volume and have a maximum size of 65,535 tracks. Besides members, a PDS contains also a directory. Each member can be accessed indirectly via the directory structure. Once a member is located, the data stored in that member are handled in the same manner as a PS (sequential) data set. Whenever a member is deleted, the space it occupied is unusable for storing other data. Likewise, if a member is re-written, it is stored in a new spot at the back of the PDS and leaves wasted “dead” space in the middle. The only way to recover “dead” space is to perform file compression. Compression, which is done using the IEBCOPY utility, moves all members to the front of the data space and leaves free usable space at the back. (Note that in modern parlance, this kind of operation might be called defragmentation or garbage collection; data compression nowadays refers to a different, more complicated concept.) PDS files can only reside on DASD, not on magnetic tape, in order to use the directory structure to access individual members. Partitioned data sets are most often used for storing multiple job control language files, utility control statements, and executable modules. An improvement of this scheme is a Partitioned Data Set Extended (PDSE or PDS/E, sometimes just libraries) introduced with DFSMSdfp for MVS/XA and MVS/ESA systems. A PDS/E library can store program objects or other types of members, but not both. BPAM cannot process a PDS/E containing program objects. PDS/E structure is similar to PDS and is used to store the same types of data. However, PDS/E files have a better directory structure which does not require pre-allocation of directory blocks when the PDS/E is defined (and therefore does not run out of directory blocks if not enough were specified). Also, PDS/E automatically stores members in such a way that compression operation is not needed to reclaim "dead" space. PDS/E files can only reside on DASD in order to use the directory structure to access individual members. == Generation Data Group == A Generation Data Group (GDG) is a group of non-VSAM data sets that are successive generations of historically-related data stored on an IBM mainframe (running OS/360 and its successors or DOS/360 and its successors). A GDG is usually cataloged. An individual member of the GDG collection is called a "Generation Data Set." The latter may be identified by an absolute number, ACCTG.OURGDG(1234), or a relative number: (-1) for the previous generation, (0) for the current one, and (+1) the next generation. A GDG specifies how many generations of a data set are to be kept and at what age a generation will be deleted. Whenever a new generation is created, the system checks whether one or more obsolete generations are to be deleted. The purpose of GDGs is to automate archival, using the command language JCL, the data set name given is generic. When DSN appears, the GDG data set appears along with the history number, where (0) is the most recent version (-1), (-2), ... are previous generations (+1) a new generation (see DD) Another use of GDGs is to be able to address all generations simultaneously within a JCL script without having to know the number of currently available generations. To do this, you have to omit the parentheses and the generation number in the JCL when specifying the dataset. === GDG JCL & features === Generation Data Groups are defined using either the BLDG statement of the IEHPROGM utility or the DEFINE GENERATIONGROUP statement of the newer IDCAMS utility, which allows setting various parameters. LIMIT(10) would limit the number of generations limit to 10. SCRATCH FOR (91) would retain each member, up to the limited#generations, at least 91 days. IDCAMS can also delete (and optionally uncatalog) a GDG. ==== Example ==== Creation of a standard GDG for five safety scopes, each at least 35 days old: Delete a standard GDG:

  Read more →
• 

  Torus interconnect

  A torus interconnect is a switch-less network topology for connecting processing nodes in a parallel computer system. == Introduction == In geometry, a torus is created by revolving a circle about an axis coplanar to the circle. While this is a general definition in geometry, the topological properties of this type of shape describes the network topology in its essence. === Geometry illustration === In the representations below, the first is a one dimension torus, a simple circle. The second is a two dimension torus, in the shape of a 'doughnut'. The animation illustrates how a two dimension torus is generated from a rectangle by connecting its two pairs of opposite edges. At one dimension, a torus topology is equivalent to a ring interconnect network, in the shape of a circle. At two dimensions, it becomes equivalent to a two dimension mesh, but with extra connection at the edge nodes. === Torus network topology === A torus interconnect is a switch-less topology that can be seen as a mesh interconnect with nodes arranged in a rectilinear array of N = 2, 3, or more dimensions, with processors connected to their nearest neighbors, and corresponding processors on opposite edges of the array connected.[1] In this lattice, each node has 2N connections. This topology is named for the lattice formed in this way, which is topologically homogeneous to an N-dimensional torus. == Visualization == The first 3 dimensions of torus network topology are easier to visualize and are described below: 1D Torus: one dimension, n nodes are connected in closed loop with each node connected to its two nearest neighbors. Communication can take place in two directions, +x and −x. A 1D Torus is the same as ring interconnection. 2D Torus: two dimensions with degree of four, the nodes are imagined laid out in a two-dimensional rectangular lattice of n rows and n columns, with each node connected to its four nearest neighbors, and corresponding nodes on opposite edges connected. Communication can take place in four directions, +x, −x, +y, and −y. The total nodes of a 2D Torus is n2. 3D Torus: three dimensions, the nodes are imagined in a three-dimensional lattice in the shape of a rectangular prism, with each node connected with its six neighbors, with corresponding nodes on opposing faces of the array connected. Each edge consists of n nodes. communication can take place in six directions, +x, −x, +y, −y, +z, −z. Each edge of a 3D Torus consist of n nodes. The total nodes of 3D Torus is n3. ND Torus: N dimensions, each node of an N dimension torus has 2N neighbors, Communication can take place in 2N directions. Each edge consists of n nodes. Total nodes of this torus is nN. The main motivation of having higher dimension of torus is to achieve higher bandwidth, lower latency, and higher scalability. Higher-dimensional arrays are difficult to visualize. The above ruleset shows that each higher dimension adds another pair of nearest neighbor connections to each node. == Performance == A number of supercomputers on the TOP500 list use three-dimensional torus networks, e.g. IBM's Blue Gene/L and Blue Gene/P, and the Cray XT3. IBM's Blue Gene/Q uses a five-dimensional torus network. Fujitsu's K computer and the PRIMEHPC FX10 use a proprietary three-dimensional torus 3D mesh interconnect called Tofu. === 3D Torus performance simulation === Sandeep Palur and Dr. Ioan Raicu from Illinois Institute of Technology conducted experiments to simulate 3D torus performance. Their experiments ran on a computer with 250GB RAM, 48 cores and x86_64 architecture. The simulator they used was ROSS (Rensselaer’s Optimistic Simulation System). They mainly focused on three aspects: Varying network size Varying number of servers Varying message size They concluded that throughput decreases with the increase of servers and network size. Otherwise, throughput increases with the increase of message size. === 6D Torus product performance === Fujitsu Limited developed a 6D torus computer model called "Tofu". In their model, a 6D torus can achieve 100 GB/s off-chip bandwidth, 12 times higher scalability than a 3D torus, and high fault tolerance. The model is used in the K computer and Fugaku. === Cost === While long wrap-around links may be the easiest way to visualize the connection topology, in practice, restrictions on cable lengths often make long wrap-around links impractical. Instead, directly connected nodes—including nodes that the above visualization places on opposite edges of a grid, connected by a long wrap-around link—are physically placed nearly adjacent to each other in a folded torus network. Every link in the folded torus network is very short—almost as short as the nearest-neighbor links in a simple grid interconnect—and therefore low-latency.

  Read more →