Structured sparsity regularization

Structured sparsity regularization

Structured sparsity regularization is a class of methods, and an area of research in statistical learning theory, that extend and generalize sparsity regularization learning methods. Both sparsity and structured sparsity regularization methods seek to exploit the assumption that the output variable Y {\displaystyle Y} (i.e., response, or dependent variable) to be learned can be described by a reduced number of variables in the input space X {\displaystyle X} (i.e., the domain, space of features or explanatory variables). Sparsity regularization methods focus on selecting the input variables that best describe the output. Structured sparsity regularization methods generalize and extend sparsity regularization methods, by allowing for optimal selection over structures like groups or networks of input variables in X {\displaystyle X} . Common motivation for the use of structured sparsity methods are model interpretability, high-dimensional learning (where dimensionality of X {\displaystyle X} may be higher than the number of observations n {\displaystyle n} ), and reduction of computational complexity. Moreover, structured sparsity methods allow to incorporate prior assumptions on the structure of the input variables, such as overlapping groups, non-overlapping groups, and acyclic graphs. Examples of uses of structured sparsity methods include face recognition, magnetic resonance image (MRI) processing, socio-linguistic analysis in natural language processing, and analysis of genetic expression in breast cancer. == Definition and related concepts == === Sparsity regularization === Consider the linear kernel regularized empirical risk minimization problem with a loss function V ( y i , f ( x ) ) {\displaystyle V(y_{i},f(x))} and the ℓ 0 {\displaystyle \ell _{0}} "norm" as the regularization penalty: min w ∈ R d 1 n ∑ i = 1 n V ( y i , ⟨ w , x i ⟩ ) + λ ‖ w ‖ 0 , {\displaystyle \min _{w\in \mathbb {R} ^{d}}{\frac {1}{n}}\sum _{i=1}^{n}V(y_{i},\langle w,x_{i}\rangle )+\lambda \|w\|_{0},} where x , w ∈ R d {\displaystyle x,w\in \mathbb {R^{d}} } , and ‖ w ‖ 0 {\displaystyle \|w\|_{0}} denotes the ℓ 0 {\displaystyle \ell _{0}} "norm", defined as the number of nonzero entries of the vector w {\displaystyle w} . f ( x ) = ⟨ w , x i ⟩ {\displaystyle f(x)=\langle w,x_{i}\rangle } is said to be sparse if ‖ w ‖ 0 = s < d {\displaystyle \|w\|_{0}=s 0 {\displaystyle w_{j}>0} . However, as in this case groups may overlap, we take the intersection of the complements of those groups that are not set to zero. This intersection of complements selection criteria implies the modeling choice that we allow some coefficients within a particular group g {\displaystyle g} to be set to zero, while others within the same group g {\displaystyle g} may remain positive. In other words, coefficients within a group may differ depending on the several group memberships that each variable within the group may have. ==== Union of groups: latent group Lasso ==== A different approach is to consider union of groups for variable selection. This approach captures the modeling situation where variables can be selected as long as they belong at least to one group with positive coefficients. This modeling perspective implies that we want to preserve group structure. The formulation of the union of groups approach is also referred to as latent group Lasso, and requires to modify the group ℓ 2 {\displaystyle \ell _{2}} norm considered above and introduce the following regularizer R ( w ) = i n f { ∑ g ‖ w g ‖ g : w = ∑ g = 1 G w ¯ g } {\displaystyle R(w)=inf\left\{\sum _{g}\|w_{g}\|_{g}:w=\sum _{g=1}^{G}{\bar {w}}_{g}\right\}} where w ∈ R d {\displaystyle w\in {\mathbb {R^{d}} }} , w g ∈ G g {\displaystyle w_{g}\in G_{g}} is the vector of coefficients of group g, and w ¯ g ∈ R d {\displaystyle {\bar {w}}_{g}\in {\mathbb {R^{d}} }} is a vector with coefficients w g j {\displaystyle w_{g}^{j}} for all variables j {

Decorrelation

Decorrelation is a general term for any process that is used to reduce autocorrelation within a signal, or cross-correlation within a set of signals, while preserving other aspects of the signal. A frequently used method of decorrelation is the use of a matched linear filter to reduce the autocorrelation of a signal as far as possible. Since the minimum possible autocorrelation for a given signal energy is achieved by equalising the power spectrum of the signal to be similar to that of a white noise signal, this is often referred to as signal whitening. == Process == === Signal processing === Most decorrelation algorithms are linear, but there are also non-linear decorrelation algorithms. Many data compression algorithms incorporate a decorrelation stage. For example, many transform coders first apply a fixed linear transformation that would, on average, have the effect of decorrelating a typical signal of the class to be coded, prior to any later processing. This is typically a Karhunen–Loève transform, or a simplified approximation such as the discrete cosine transform. By comparison, sub-band coders do not generally have an explicit decorrelation step, but instead exploit the already-existing reduced correlation within each of the sub-bands of the signal, due to the relative flatness of each sub-band of the power spectrum in many classes of signals. Linear predictive coders can be modelled as an attempt to decorrelate signals by subtracting the best possible linear prediction from the input signal, leaving a whitened residual signal. Decorrelation techniques can also be used for many other purposes, such as reducing crosstalk in a multi-channel signal, or in the design of echo cancellers. In image processing decorrelation techniques can be used to enhance or stretch, colour differences found in each pixel of an image. This is generally termed as 'decorrelation stretching'. === Neuroscience === In neuroscience, decorrelation is used in the analysis of the neural networks in the human visual system. The raw inputs from cone cells and rod cells under go many steps of processing before it is handled by the visual cortex. These steps generally perform decorrelation, both spatial (surround suppression in the retina) and temporal (handling of movement in the lateral geniculate nucleus). === Cryptography === In cryptography, decorrelation is used in cipher design (see Decorrelation theory) and in the design of hardware random number generators.

FutureMedia

FutureMedia is a program that analyzes the state and future of digital, social, and mobile media. It functions as a collaborative initiative at Georgia Tech and the Georgia Tech Research Institute. FutureMedia consults approximately 500 faculty members working in those fields. == History == In 2019, Future Media expanded into the Direct-To-Consumer market by acquiring Australian watchmaker Oak & Jackal. == Programs == === FutureMedia Fest === The organization most recently hosted FutureMedia Fest 2010, a four-day conference (Oct 4–7, 2010) with a keynote addresses from Michael Jones, the chief technology advocate at Google. The event featured panels, workshops, and technology demonstrations. === FutureMedia Outlook === Contemporaneous with FutureMedia Fest 2010, the organization released the FutureMedia Outlook, an analysis of the future of media, concentrating on six major trends in those fields, including information overload, personalization, data integrity, an expectation of multimedia, augmented reality, and collaborative software.

Raseef22

Raseef22 (Arabic: رصيف22) is a liberal Arabic media network founded in 2013 based in Beirut, Lebanon. It publishes content in Arabic and English from different Arab states and describes itself as an independent media platform. International Media Support mentions Raseef22 along with HuffPost Arabic and Al Jazeera as one of the biggest Pan-Arab online platforms. == Name == The Arabic word raseef (رَصِيف) means platform or pavement, and the number 22 refers to the number of states in the Arab League. == History == Kareem Sakka co-founded Raseef22 in the aftermath of the Arab Spring, which he cites as a source of inspiration. In an article in The Washington Post, he wrote that Raseef22 was created as a "digital space for those eager to know what was going on around them." Raseef22 was one of the 500 websites censored in Egypt in late 2017 after it published an article on Egyptian security agencies' vies to influence the media. After the site was blocked in Egypt, it was targeted in a cyber attack that took it offline in locations around the world. Jamal Khashoggi wrote for Raseef22 regularly. One of his notable articles was "Notes on the Freedom of the Arabs from Oslo, Norway," published June 5, 2018. The site was blocked in Saudi Arabia December 2018 when the Saudi Ministry of Communications and Information Technology ordered its censorship due to its "unprecedented response to the assassination of Jamal Khashoggi in Istanbul." This decision might have also been related to Raseef22's coverage of Saudi-Israeli relations and interviews with activists later imprisoned or placed under house arrest coverage In 2019 the Association of LGBT Journalists (AJL) in Paris gave Raseef22 a golden foreign press award for its six-month series of articles on gender and sexuality issues. == Readership == According to its publisher in 2019, the news agency counted 12 million readers annually from 22 Arab nations. Of the readership, he wrote that it "believes in the talent and promise of the Arab mind and sees the ugliness of tyranny, patriarchy, misogyny and the futility of proxy rulers and wars." Al-Quds Al-Arabi described Raseef22 as "oriented to the youth."

Modulation error ratio

The modulation error ratio (MER) is a measure used to quantify the performance of a digital radio (or digital TV) transmitter or receiver in a communications system using digital modulation (such as QAM). A signal sent by an ideal transmitter or received by a receiver would have all constellation points precisely at the ideal locations, however various imperfections in the implementation (such as noise, low image rejection ratio, phase noise, carrier suppression, distortion, etc.) or signal path cause the actual constellation points to deviate from the ideal locations. Transmitter MER can be measured by specialized equipment, which demodulates the received signal in a similar way to how a real radio demodulator does it. Demodulated and detected signal can be used as a reasonably reliable estimate for the ideal transmitted signal in MER calculation. == Definition == An error vector is a vector in the I-Q plane between the ideal constellation point and the point received by the receiver. The Euclidean distance between the two points is its magnitude. The modulation error ratio is equal to the ratio of the root mean square (RMS) power (in Watts) of the reference vector to the power (in Watts) of the error. It is defined in dB as: M E R ( d B ) = 10 log 10 ⁡ ( P s i g n a l P e r r o r ) {\displaystyle \mathrm {MER(dB)} =10\log _{10}\left({P_{\mathrm {signal} } \over P_{\mathrm {error} }}\right)} where Perror is the RMS power of the error vector, and Psignal is the RMS power of ideal transmitted signal. MER is defined as a percentage in a compatible (but reciprocal) way: M E R ( % ) = P e r r o r P s i g n a l × 100 % {\displaystyle \mathrm {MER(\%)} ={\sqrt {P_{\mathrm {error} } \over P_{\mathrm {signal} }}}\times 100\%} with the same definitions. MER is closely related to error vector magnitude (EVM), but MER is calculated from the average power of the signal. MER is also closely related to signal-to-noise ratio. MER includes all imperfections including deterministic amplitude imbalance, quadrature error and distortion, while noise is random by nature.

Machine-learned interatomic potential

Machine-learned interatomic potentials (MLIPs), or simply machine learning potentials (MLPs), are interatomic potentials constructed using machine learning. Beginning in the 1990s, researchers have employed such programs to construct interatomic potentials by mapping atomic structures to their potential energies. These potentials are referred to as MLIPs or MLPs. Such machine learning potentials promised to fill the gap between density functional theory, a highly accurate but computationally intensive modelling method, and empirically derived or intuitively-approximated potentials, which were far lighter computationally but substantially less accurate. Improvements in artificial intelligence technology heightened the accuracy of MLPs while lowering their computational cost, increasing the role of machine learning in fitting potentials. Machine learning potentials began by using neural networks to tackle low-dimensional systems. While promising, these models could not systematically account for interatomic energy interactions; they could be applied to small molecules in a vacuum, or molecules interacting with frozen surfaces, but not much else – and even in these applications, the models often relied on force fields or potentials derived empirically or with simulations. These models thus remained confined to academia. Modern neural networks construct highly accurate and computationally light potentials, as theoretical understanding of materials science was increasingly built into their architectures and preprocessing. Almost all are local, accounting for all interactions between an atom and its neighbor up to some cutoff radius. There exist some nonlocal models, but these have been experimental for almost a decade. For most systems, reasonable cutoff radii enable highly accurate results. Almost all neural networks intake atomic coordinates and output potential energies. For some, these atomic coordinates are converted into atom-centered symmetry functions. From this data, a separate atomic neural network is trained for each element; each atomic network is evaluated whenever that element occurs in the given structure, and then the results are pooled together at the end. This process – in particular, the atom-centered symmetry functions which convey translational, rotational, and permutational invariances – has greatly improved machine learning potentials by significantly constraining the neural network search space. Other models use a similar process but emphasize bonds over atoms, using pair symmetry functions and training one network per atom pair. Other models to learn their own descriptors rather than using predetermined symmetry-dictating functions. These models, called message-passing neural networks (MPNNs), are graph neural networks. Treating molecules as three-dimensional graphs (where atoms are nodes and bonds are edges), the model takes feature vectors describing the atoms as input, and iteratively updates these vectors as information about neighboring atoms is processed through message functions and convolutions. These feature vectors are then used to predict the final potentials. The flexibility of this method often results in stronger, more generalizable models. In 2017, the first-ever MPNN model (a deep tensor neural network) was used to calculate the properties of small organic molecules. == Gaussian Approximation Potential (GAP) == One popular class of machine-learned interatomic potential is the Gaussian Approximation Potential (GAP), which combines compact descriptors of local atomic environments with Gaussian process regression to machine learn the potential energy surface of a given system. To date, the GAP framework has been used to successfully develop a number of MLIPs for various systems, including for elemental systems such as carbon, silicon, phosphorus, and tungsten, as well as for multicomponent systems such as Ge2Sb2Te5 and austenitic stainless steel, Fe7Cr2Ni. == Equivariant graph neural networks == A significant limitation of early MPNNs was that they were not inherently equivariant to rotations and reflections of atomic structures — meaning predictions could change depending on how a molecule was oriented in space. Beginning around 2021, a new class of models addressed this by incorporating equivariance directly into the message-passing layers using spherical harmonics and irreducible representations. Notable examples include NequIP (2021), MACE (2022), and GemNet-OC (2022). These equivariant architectures proved substantially more data-efficient and accurate than their predecessors, and became the dominant paradigm for high-accuracy MLIPs. == Universal MLIPs and large-scale datasets == Early MLIPs were system-specific, trained on a few thousand structures of a single material. A major shift occurred with the creation of large, chemically diverse datasets enabling models that generalize across many elements, bonding environments, and application domains — so-called universal MLIPs. A key driver was the Open Catalyst Project (OC20, OC22), a collaboration between Meta AI (FAIR) and Carnegie Mellon University launched in 2020. OC20 comprises approximately 1.3 million DFT relaxations across 82 elements, designed to accelerate the discovery of catalysts for renewable energy applications. It was among the first datasets large enough to train GNNs that generalize across diverse chemical systems, and established a widely-used benchmark for the field. A subsequent dataset, Open Direct Air Capture (OpenDAC 2023 and OpenDAC 2025), applied the same approach to carbon capture, providing a large computational database of metal-organic frameworks and sorbent candidates evaluated for CO₂ capture, generated using nearly 400 million CPU hours of quantum chemistry calculations in collaboration with Georgia Tech. These datasets revealed a new challenge: the GNN architectures most effective for atomic simulations were memory-intensive, as they model higher-order interactions between triplets or quadruplets of atoms, making it difficult to scale model size. Graph Parallelism, introduced by Sriram et al. (ICLR 2022), addressed this by distributing a single input graph across multiple GPUs — a distinct strategy from data parallelism (which distributes training examples) or model parallelism (which distributes layers). This enabled training GNNs with hundreds of millions to billions of parameters for the first time. Building on these foundations, Meta FAIR released the Universal Model for Atoms (UMA) in 2025, trained on approximately 500 million unique 3D atomic structures spanning molecules, materials, and catalysts — the largest training run to date for an MLIP. UMA introduced a Mixture of Linear Experts (MoLE) architecture, enabling one model to learn from datasets generated by different DFT codes and settings without significant inference overhead. It matches or surpasses specialized models across catalysis, materials, and molecular benchmarks without task-specific fine-tuning, and has been described as marking a "pre/post-UMA" divide in the field. == Applications == Catalyst discovery: MLIPs have significantly accelerated the computational screening of heterogeneous catalysts by replacing expensive DFT relaxations with fast neural network surrogates. The Open Catalyst Project explicitly targets this application, aiming to identify new catalysts for green hydrogen production and other renewable energy reactions. Carbon capture: The OpenDAC project applies universal MLIPs to screening sorbent materials for direct air capture of CO₂, a key technology for climate change mitigation. AI-accelerated screening allows evaluation of orders of magnitude more candidate materials than traditional DFT workflows. Drug discovery and molecular design: MLIPs are increasingly used in pharmaceutical research to model molecular conformations and binding energies. The Open Molecules 2025 (OMol25) dataset, released by Meta FAIR in 2025, provides high-accuracy calculations for a large set of molecular systems to support this use case. Materials discovery: Universal MLIPs enable high-throughput screening of novel inorganic materials, including battery electrolytes, semiconductors, and superconductors, by rapidly estimating stability and properties across large chemical spaces.

Optical recording

The history of optical recording can be divided into a few number of distinct major contributions. The pioneers of optical recording worked mostly independently, and their solutions to the many technical challenges have very distinctive features, such as reflective disc (Compaan and Kramer) transparent disc (Gregg) floppy disc (Russell) rigid disc (Compaan and Kramer) focused laser beam for read-out through transparent substrate (Compaan and Kramer). == Gregg 1958 == Laserdisc technology, using a transparent disc, was invented by David Paul Gregg in 1958 (and patented in 1970 and 1990). By 1969 Philips had developed a videodisc in reflective mode, which has great advantages over the transparent mode. MCA and Philips decided to join their efforts. They first publicly demonstrated the videodisc in 1972. Laserdisc was first available on the market, in Atlanta, on December 15, 1978, two years after the VHS VCR and four years before the CD, which is based on Laserdisc technology. Philips produced the players and MCA produced the discs. The Philips/MCA cooperation was not successful, and discontinued after a few years. Several of the scientists responsible for the early research (John Winslow, Richard Wilkinson and Ray Dakin) founded Optical Disc Corporation (now ODC Nimbus). == Russell 1965 == While working at Pacific Northwest National Laboratory, James Russell invented an optical storage system for digital audio and video, patenting the concept in 1970. The earliest patents by Russell, US 3,501,586, and 3,795,902 were filed in 1966, and 1969. respectively. He built prototypes, and the first was operating in 1973. Russell had found a way to record digital information onto a photosensitive plate in tiny dark spots, each spot one micrometre from centre to centre, with a laser that wrote the binary patterns. Russell's first optical disc was distinctly different from the eventual compact disc product: the disc in the player was not read by laser light. A key characteristic of Russell's invention is that a laser is not used for the reading the disc, instead the entire disc or oblong sheet to be read is illuminated by a large playback light source at the back of the transparent foil. As a result, the information density is relatively low. By 1985, Russell held over 25 patents to various technologies related to optical recording and playback. Russell's intellectual property was purchased by Optical Recording Corporation (ORC) in Toronto in 1985, and this firm notified a number of CD manufacturers that their CD technology was based on patents held by ORC. In 1987, ORC signed an agreement with Sony whereby Sony paid for licensing of the technology. Further licenses followed from Philips and others. Warner Communications did not sign, and was sued by ORC. In 1992, the large CD manufacturer, now called Time Warner, was ordered to pay ORC US$30 million in patent violations. In the 1970 patent, the spot diameter was around 10 micrometres. Thus, the areal information density was around a factor hundred less than that of the CD as later developed. Russell continued to refine the concept throughout the 1970s. Philips and Sony, however, were able to put far greater resources into the parallel development of the concept, arriving at a smaller and more sophisticated product in just a few years. Russell's various partners and ventures failed to produce a single consumer product. == Korpel 1968 == Adrianus Korpel worked for the Zenith Electronics Corporation, when he developed very early optical videodisc systems, including holographic storage. == Kramer and Compaan 1969 == The Philips development of the videodisc technology began in 1969 with efforts by Dutch physicists Klaas Compaan and Piet Kramer to record video images in holographic form on disc. Their prototype Laserdisc shown in 1972 used a laser beam in reflective mode to read a track of pits using an FM video signal. Together with MCA, Philips brought the optical videodisk to market in 1978. The cooperation between Philips and MCA did not last long, and discontinued after a few years. == Immink and Doi 1979 == The Compact Disc (CD), which is based on MCA/Philips Laserdisc technology, was developed by a taskforce of Sony and Philips in 1979–1980. Toshi Doi and Kees Schouhamer Immink created the digital technologies that turned the analog Laserdisc into a high-density low-cost digital audio disc. The CD, available on the market since October 1982, remains the standard physical medium for sale of commercial audio recordings Standard CDs have a diameter of 120 mm and can hold up to 80 minutes of audio (700 MB of data). The Mini CD has various diameters ranging from 60 to 80 mm; they are sometimes used for CD singles or device drivers, storing up to 24 minutes of audio. The technology was later adapted and expanded to include data storage CD-ROM, write-once audio and data storage CD-R, rewritable media CD-RW, Super Audio CD (SACD), Video Compact Discs (VCD), Super Video Compact Discs (SVCD), PhotoCD, PictureCD, CD-i, and Enhanced CD. CD-ROMs and CD-Rs remain widely used technologies in the computer industry. The CD and its extensions have been extremely successful: in 2004, worldwide sales of CD audio, CD-ROM, and CD-R reached about 30 billion discs. By 2007, 200 billion CDs had been sold worldwide.