Discrete diffusion model

In machine learning, discrete diffusion models are a class of diffusion models, which themselves are a class of latent variable generative models. Each discrete diffusion model consists of two major components: the forward jump diffusion process, and the reverse jump diffusion process. The goal of diffusion modeling is, given a given dataset and a forward process, to learn a model for the reverse process, such that the reverse process can generate new elements that are distributed similarly as the original dataset. A trained discrete diffusion model can be sampled in many ways, which trades off computational efficiency and sample quality. In general, higher quality data can be obtained, but at the price of higher computational cost. In standard diffusion modeling, the diffusion process takes place over a state space that is continuous space of R n {\displaystyle \mathbb {R} ^{n}} , but over a discrete set S {\displaystyle S} . A discrete set is simply a set where one cannot speak of "infinitesimally close" points. Points can be more or less separated from each other, but the separation is always a finite number. This in particular means the standard framework of continuous diffusion does not apply, since it uses gaussian noise, which is continuous. Nevertheless, an analogous theory can be produced. Discrete diffusion is usually used for language modeling. In practice, the state space S {\displaystyle S} is not only discrete, but finite, so this is what we will assume from now on. == Continuous time Markov process == In the case of continuous state space, during the forward discrete diffusion process, at each step t → t + d t {\displaystyle t\to t+dt} , we mix in an infinitesimal amount of gaussian noise d x t = − 1 2 β ( t ) x t d t + β ( t ) d W t {\displaystyle dx_{t}=-{\frac {1}{2}}\beta (t)x_{t}dt+{\sqrt {\beta (t)}}dW_{t}} . This changes the probability density function, by first a convolution with the density of a gaussian, followed by a scaling. In the case of discrete state space, the gaussian noise must be replaced by a noise that takes values over a finite set. For example, if the noise is the uniform distribution over S {\displaystyle S} , then the probability distribution at time t + d t {\displaystyle t+dt} satisfies q t + d t ( x ) = ( 1 − d t ) q t ( x ) + d t ( 1 | S | ∑ y ∈ S q t ( y ) ) {\displaystyle q_{t+dt}(x)=(1-dt)q_{t}(x)+dt\left({\frac {1}{|S|}}\sum _{y\in S}q_{t}(y)\right)} More succinctly, ∂ t q t ( x ) = − ( 1 − 1 | S | ) q t ( x ) + ∑ y ∈ S , y ≠ x 1 | S | q t ( y ) {\displaystyle \partial _{t}q_{t}(x)=-\left(1-{\frac {1}{|S|}}\right)q_{t}(x)+\sum _{y\in S,y\neq x}{\frac {1}{|S|}}q_{t}(y)} In general, we do not need to convolve with a uniformly distributed noise, but with an arbitrary noise process. That is, we use an arbitrary matrix Q t {\displaystyle Q_{t}} such that ∂ t q t ( y ) = ∑ x ∈ S Q t ( y , x ) q t ( x ) {\displaystyle \partial _{t}q_{t}(y)=\sum _{x\in S}Q_{t}(y,x)q_{t}(x)} where Q t {\displaystyle Q_{t}} is called the rate matrix. Any matrix may be used as a rate matrix if it has non-negative off-diagonals, and each column sums to 0: Q t ( y , x ) ≥ 0 ∀ y ≠ x , ∑ y ∈ S Q t ( y , x ) = 0 ∀ x {\displaystyle Q_{t}(y,x)\geq 0\quad \forall y\neq x,\quad \sum _{y\in S}Q_{t}(y,x)=0\quad \forall x} A continuous time Markov chain (CTMC) is defined by a continuous function Q {\displaystyle Q} that maps any time t ∈ [ 0 , T ) {\displaystyle t\in [0,T)} to a rate matrix Q t {\displaystyle Q_{t}} . Given the function Q {\displaystyle Q} , time-evolution under the CTMC is done as follows: Given state x t {\displaystyle x_{t}} at time t {\displaystyle t} , and given an infinitesimal d t {\displaystyle dt} , the state at t + d t {\displaystyle t+dt} is x t + d t {\displaystyle x_{t+dt}} , such that Pr ( x t + d t | x t ) = { 1 + Q t ( x t + d t , x t ) d t if x t + d t = x t Q t ( x t + d t , x t ) d t else {\displaystyle \Pr(x_{t+dt}|x_{t})={\begin{cases}1+Q_{t}(x_{t+dt},x_{t})dt&{\text{if }}x_{t+dt}=x_{t}\\Q_{t}(x_{t+dt},x_{t})dt&{\text{else}}\end{cases}}} This implies that the probability distribution function evolves according to ∂ t q t ( y ) = ∑ x ∈ S Q t ( y , x ) q t ( x ) {\displaystyle \partial _{t}q_{t}(y)=\sum _{x\in S}Q_{t}(y,x)q_{t}(x)} which is what we previously specified. === Backward process === Similarly to the case of continuous diffusion, in discrete diffusion, there exists a backward diffusion process Q ¯ t {\displaystyle {\bar {Q}}_{t}} : s ( x , t ) y := q t ( y ) q t ( x ) , Q ¯ t ( y , x ) := { s ( x , t ) y Q t ( x , y ) if y ≠ x − ∑ y : y ≠ x Q ¯ t ( y , x ) if y = x {\displaystyle s(x,t)_{y}:={\frac {q_{t}(y)}{q_{t}(x)}},\quad {\bar {Q}}_{t}(y,x):={\begin{cases}s(x,t)_{y}Q_{t}(x,y)&{\text{if }}y\neq x\\-\sum _{y:y\neq x}{\bar {Q}}_{t}(y,x)&{\text{if }}y=x\end{cases}}} where s ( x , t ) y {\displaystyle s(x,t)_{y}} should be interpreted as the discrete score or concrete score, since, abusing notation a bit, the score function is ∇ ln ⁡ ρ t ( x ) = 1 d x ( ρ t ( x + d x ) ρ t ( x ) − 1 ) {\displaystyle \nabla \ln \rho _{t}(x)={\frac {1}{dx}}\left({\frac {\rho _{t}(x+dx)}{\rho _{t}(x)}}-1\right)} . If we picture the distribution q t {\displaystyle q_{t}} as a bunch of point-masses, one per state x ∈ S {\displaystyle x\in S} , then the forward diffusion from time t {\displaystyle t} to t + d t {\displaystyle t+dt} is performed by removing Q t ( x , y ) q t ( y ) d t {\displaystyle Q_{t}(x,y)q_{t}(y)dt} from the mass at y {\displaystyle y} and moving it to the mass at x {\displaystyle x} , for each pair x ≠ y {\displaystyle x\neq y} . Thus, the process is reversed in detail by the CTMC defined by Q ¯ {\displaystyle {\bar {Q}}} , since Q ¯ t ( y , x ) q t ( x ) = Q t ( x , y ) q t ( y ) {\displaystyle {\bar {Q}}_{t}(y,x)q_{t}(x)=Q_{t}(x,y)q_{t}(y)} . Given Q ¯ t {\displaystyle {\bar {Q}}_{t}} , if we have a way to sample from q t {\displaystyle q_{t}} , then we can sample from q t − d t {\displaystyle q_{t-dt}} by first sampling x t ∼ q t {\displaystyle x_{t}\sim q_{t}} , then sampling x t − d t {\displaystyle x_{t-dt}} according to Pr ( x t − d t | x t ) = { 1 + Q ¯ t ( x t − d t , x t ) d t if x t − d t = x t Q ¯ t ( x t − d t , x t ) d t else {\displaystyle \Pr(x_{t-dt}|x_{t})={\begin{cases}1+{\bar {Q}}_{t}(x_{t-dt},x_{t})dt&{\text{if }}x_{t-dt}=x_{t}\\{\bar {Q}}_{t}(x_{t-dt},x_{t})dt&{\text{else}}\end{cases}}} === Overall plan of score-matching discrete diffusion modeling === Similar to score-matching continuous diffusion, score-matching discrete diffusion is a method to sample an initial distribution. If we have a certain function s θ {\displaystyle s_{\theta }} that approximates the true score function s θ ( x , t ) y ≈ s ( x , t ) y {\displaystyle s_{\theta }(x,t)_{y}\approx s(x,t)_{y}} , then it allows a corresponding Q ¯ θ {\displaystyle {\bar {Q}}^{\theta }} to be defined in the same way. If we also have a base distribution q base {\displaystyle q_{\text{base}}} such that it is easy to sample from, and approximately equal to the true terminal distribution q base ≈ q T {\displaystyle q_{\text{base}}\approx q_{T}} , then we can perform the backward CTMC with Q ¯ θ {\displaystyle {\bar {Q}}^{\theta }} and q T θ := q terminal {\displaystyle q_{T}^{\theta }:=q_{\text{terminal}}} . When both approximations are good, the backward CTMC would give q 0 θ ≈ q 0 {\displaystyle q_{0}^{\theta }\approx q_{0}} . This is the idea of score-matching discrete diffusion modeling. If q data {\displaystyle q_{\text{data}}} is sharp, in the sense that for some x , x ′ {\displaystyle x,x'} , we have q data ( x ) ≫ q data ( x ′ ) {\displaystyle q_{\text{data}}(x)\gg q_{\text{data}}(x')} , then the score function would diverge as 1 / t {\displaystyle 1/t} at the t → 0 {\displaystyle t\to 0} limit. To avoid this in practice, it is common to use early stopping, which is to stop the backward process at some time δ > 0 {\displaystyle \delta >0} , and sample from q δ θ {\displaystyle q_{\delta }^{\theta }} instead of q 0 θ {\displaystyle q_{0}^{\theta }} . === Tractable forward processes === The theory of CTMC works for any continuous choice of rate matrices Q {\displaystyle Q} . However, most choices are computationally expensive and cannot be used in practice. In the case of continuous diffusion, the gaussian noise is used for the simple reason that the sum of any number of gaussians is still a gaussian. This allows one to sample any x t ∼ ρ t {\displaystyle x_{t}\sim \rho _{t}} by sampling a single x 0 ∼ ρ 0 {\displaystyle x_{0}\sim \rho _{0}} , followed by a single gaussian noise z ∼ N ( 0 , I ) {\displaystyle z\sim {\mathcal {N}}(0,I)} , and let x t = α ¯ t x 0 + σ t z {\displaystyle x_{t}={\sqrt {{\bar {\alpha }}_{t}}}x_{0}+\sigma _{t}z} , without needing any x s {\displaystyle x_{s}} for any 0 < s < t {\displaystyle 0

Dyme (company)

Dyme is a Dutch fintech start-up and subscription management app that allows users to cancel and renegotiate their recurring costs. In 2019, Dyme was the first independent Dutch company to receive a PSD2 licence from the Netherlands' central bank (DNB). == History == Dyme was founded in 2018 by Joran Iedema, David Knap, David Schogt and Wouter Florijn. The four had previously founded Cycleswap, a bicycle rental platform launched in 2015 and sold to the American platform Spinlister in 2016. The company gained notability in the Netherlands in 2020 when it appeared on Dutch television in Dragons Den, where Pieter Schoen made a €750,000 bid in an attempt to acquire 51.01% of the company. Dyme's Joran Iedema rejected the deal. == Recognition == Wired described Dyme as one of the "hottest start-ups in Europe" in 2021. As of 2021, the company reportedly had 350,000 registered users in the Netherlands and Great Britain.

Trebel (music app)

Trebel is an on-demand music download and discovery platform developed by M&M Media Inc. The company's business model aims to combat digital music piracy by giving users access to on-demand music at no cost while delivering fair compensation to artists and music rights holders. Trebel has a patent that allows it to market itself as the only international music service in which users can legally download music and listen to it offline for free. As of March 2023, Trebel has a catalog of 75 million songs from record labels such as Universal Music Group, Sony Music Entertainment, Warner Music Group and hundreds of independent labels. Trebel is based in Stamford, Connecticut. with additional locations in Mexico City, Jakarta, Bogota, Los Angeles and Miami. The app is available in the Apple App Store, Google Play Store, and Huawei AppGallery. == History == Trebel was founded in 2014 by Gary Mekikian, who was previously the co-founder of answerFriend, Inc., which commercialized web based question-answering technologies and merged with Electric Knowledge, forming InQuira. This company was eventually acquired by Oracle Corporation in 2011. His co-founders at Trebel include Stanford classmates Corey Jones and Luis Soto Durazo, as well as his daughters Grace and Juliette. Mekikian envisioned Trebel as an alternative to music piracy after a high school classmate of his daughters was targeted by cyberattackers while illegally downloading music online. Trebel was initially released in 2015 under the name Project Carmen to students at Ohio State, Santa Monica College, Cal State Fullerton, UCLA and Long Beach State. In its original incarnation, the service planned to target students at 3,000 universities and 30,000 high schools in the United States. A beta version of the app was introduced in 2016 with content from Universal Music Group and Warner Music Group. Trebel launched commercially in the United States and Mexico in 2018. In 2018, Mexican mass-media corporation Televisa also became a minority investor in Trebel. In May 2020, during the early months of the COVID-19 pandemic, Trebel was a digital broadcast partner for Se Agradece, a concert produced in Mexico by Televisa to honor frontline COVID workers that featured artists such as Rosalia, J Balvin, Maluma and Ricky Martin. In June 2021, Trebel reached 3 million monthly active users. In October 2021, Trebel signed a music licensing agreement with Merlin Network, the licensing agency for the independent music sector that controls an estimated 12% of the global digital recorded music market. In January 2022, Trebel announced a strategic alliance with MNC Corporation, an Indonesian media conglomerate, which also became a minority backer of the company. In March 2022, Trebel reported 5.2 million monthly active users as a result of growth in Latin America. In the same month,, Latin music star Maluma became a backer of Trebel and an advisor to Gary Mekikian, helping expand the service throughout Latin America. On April 18, 2022, Trebel launched in Indonesia during the finale of the music competition show X Factor Indonesia. Trebel also signed a deal that month with Soccer Media Solutions, a sports and entertainment marketing agency in Mexico, to sell Trebel’s premium advertising inventory through Soccer Media. In May 2022, Guillermo Ochoa, goalkeeper for the Mexican national soccer team, invested in Trebel and became an ambassador for the company. On October 2, 2022, Trebel collaborated with Musica Studios, one of the largest music companies in Indonesia, on the production of a music festival in Jakarta titled Trebel Music Fest. The event featured performances by top Indonesian music artists such as Noah, Nidji, and d'Masiv. In October 2022, Trebel launched in Colombia. The service reached 1.2 million monthly active users in Colombia six months after launching. In December 2022, Trebel collaborated with KFC in Indonesia on the release of a KFC digital music program using a product called Trebel Max. As part of the program, KFC customers who bought the Crazy Superstar Combo package at KFC received a subscription to Trebel Max for 30 days. Trebel announced the launch of Trebel AI in May 2023. Trebel AI uses ChatGPT-powered technology to generate playlists based on natural language queries from users. In Indonesia, the Trebel AI feature was announced during a broadcast of the show Indonesian Idol XII that took place on May 8, 2023. In July 2023, Trebel reached more than 13 million monthly active users. In November 2023, Trebel became a featured app on the Discord app directory. Discord users that add the Trebel bot to their servers have access to Trebel's on-demand music library and have the exclusive privilege of being DJ's during server sessions with up to 150 concurrent listeners. == Platform == === Features === Trebel has a patent that allows it to market itself as the only international music service in which users can legally download music and listen to it offline for free. As of March 2023, Trebel has a catalog of 75 million songs from record labels such as Universal Music Group, Sony Music Entertainment, Warner Music Group and hundreds of independent labels. Trebel offers unlimited music downloads that are playable in the app by registered users only. Offline listening is free to all users and not blocked by a paywall. Users can search for music based on song, artist, album, browsing friends' recent activity, and through other users' playlists. The app also offers free cloud storage for downloaded songs. Trebel also contains a feature called SongID, which identifies music being played nearby using a short sample, then offers it for download on the service. Podcasts are available for free listening on the service as well. === Business model === Trebel uses a business model that generates revenue from the sale of digital advertising as well as user interactions with branded experiences, and consumption of virtual goods within the app (akin to mobile games). The app also features a brand takeover feature called Trebel Max, which offers unlimited access in exchange for users engaging with experiences offered by specific brands. Trebel’s brand partners include Uber, KFC, Walmart, Coca-Cola, Amazon and P&G. === Content === In September 2022, Trebel secured an exclusive release of the song “Suara Hatiku” by Indonesian actress Amanda Monopo. As of March 2023, Trebel offers 75 million songs through licensing agreements with Universal Music Group, Sony Music Entertainment, Warner Music Group and global indie rights agency Merlin. == Awards == In 2023, Trebel won three Google Play awards including "Best App of 2023", "Best Everyday Essentials" and "Users' Choice".

Israeli cybersecurity industry

The Israeli cybersecurity industry is a rapidly growing sector within Israel's technology and innovation ecosystem. Israel is internationally recognized as a powerhouse in the cybersecurity domain, with numerous cybersecurity startups, established companies, research institutions, and government initiatives. Tel Aviv itself is being ranked 7th in annual list of best global tech ecosystems, as reported by the Jerusalem Post. == History == The roots of Israel's cybersecurity industry can be traced back to the country's strong focus on national security and intelligence. The establishment of elite military units such as Unit 8200, the Israeli Intelligence Corps unit responsible for signals intelligence and code decryption, played a significant role in the development of cybersecurity expertise in the country. Many former members of Unit 8200 have gone on to establish successful cybersecurity companies or join existing organizations, bringing their unique skill sets and experience to the private sector. == Market overview == As of 2024, Israel housed more than 450 cybersecurity startups and companies. In 2023, the value of exits by Israeli tech companies reached $7.5 billion. Israel's cybersecurity industry is characterized by a high concentration of startups develop new technologies in areas such as network security, endpoint protection, data security, cloud security, and threat intelligence. In recent years, the sector has attracted significant investment from both local and international venture capital firms, as well as major technology companies such as Microsoft, Google, and IBM. Several Israeli cybersecurity companies have gained global recognition and success, with some being acquired by major corporations or conducting successful initial public offerings (IPOs). === Key Israeli cybersecurity companies === Some key Israeli cybersecurity companies include: Check Point Software Technologies CyberArk Cato Networks Radware Wiz === Financial activity === Israel’s cybersecurity sector has seen significant financial activity. As of 2023, mergers and acquisitions in the cybersecurity sector totaled $2.8 billion. In the first quarter of 2024, the sector secured $846 million in private funding. == Background == The military experience helped much. Israel's mandatory military service, combined with the expertise developed within elite units such as Unit 8200, has fostered a strong talent pool with practical experience in cybersecurity. Israel's thriving startup ecosystem, often referred to as the "Startup Nation," has fostered an environment of innovation and collaboration that has contributed to the growth of the cybersecurity industry. Israeli cybersecurity companies often collaborate with international partners, both in the private and public sectors, to share knowledge and develop joint solutions. === Government Initiatives and Support === The government also supported well through various initiatives, such as the Israel National Cyber Directorate (INCD), which works to strengthen cybersecurity defenses and promote the development of the sector. === Academic institutions === Israeli universities and research centers are involved in cybersecurity research and education, contributing to the development of new technologies and training the next generation of cybersecurity professionals. Academic Tech transfer offices in Israel also facilitate the commercialization of cybersecurity technologies. Some academic institutions with cybersecurity laboratories include: Tel Aviv University Technion Ben-Gurion University

Trebel (music app)

Elasticity (computing)

In computing, elasticity is defined as "the degree to which a system is able to adapt to workload changes by provisioning and de-provisioning resources in an autonomic manner, such that at each point in time the available resources match the current demand as closely as possible". Elasticity is a defining characteristic that differentiates cloud computing from previously proposed distributed computing paradigms, such as grid computing. The dynamic adaptation of capacity, e.g., by altering the use of computing resources, to meet a varying workload is called "elastic computing". In the world of distributed systems, there are several definitions according to the authors; some consider the concepts of scalability a sub-part of elasticity, others as being distinct. == Purpose == Elasticity aims to match the amount of resources allocated to a service with the amount of resources it actually requires, avoiding over- or under-provisioning. Over-provisioning, i.e., allocating more resources than required, should be avoided as it may incur extra costs (monetary, energy, operational, etc.) for unused or underutilized resources. For example, if a website is over-provisioned with two cloud computing resources to handle current demand that only requires one resource, the costs of maintaining the second resource would effectively be wasted. Under-provisioning, i.e., allocating fewer resources than required, must be avoided; otherwise, the service cannot serve its users with a good service. For example, under-provisioning a website may make it seem slow or unreachable, because not enough resources have been allocated to meet current demand. == Example == Elasticity can be illustrated through an example of a service provider who wants to run a website on the cloud. At moment t 0 {\displaystyle t_{0}} , the website is unpopular and a single machine is sufficient to serve all users. At moment t 1 {\displaystyle t_{1}} , the website suddenly becomes popular, and a single machine is no longer sufficient to serve all users. Based on the number of web users simultaneously accessing the website and the resource requirements of the web server, ten machines are needed. An elastic system should immediately detect this condition and provision nine additional machines from the cloud to serve all users responsively. At time t 2 {\displaystyle t_{2}} , the website becomes unpopular again. The ten machines currently allocated to the website are mostly idle and a single machine would be sufficient to serve the few users who are accessing the website. An elastic system should immediately detect this condition and deprovision nine machines, releasing them to the cloud. == Problems == === Resource provisioning time === Resource provisioning takes time. A cloud virtual machine (VM) can be acquired at any time by the user; however, it may take up to several minutes for the acquired VM to be ready to use. The VM startup time is dependent on factors such as image size, VM type, data center location, number of VMs, etc. Cloud providers have different VM startup performance. This implies that any control mechanism designed for elastic applications must consider the time needed for the resource provisioning actions to take effect. === Monitoring elastic applications === Elastic applications can allocate and deallocate resources on demand for specific application components. This makes cloud resources volatile, and traditional monitoring tools which associate monitoring data with a particular resource, such as Ganglia or Nagios, are no longer suitable for monitoring the behavior of elastic applications. For example, during its lifetime, a data storage tier of an elastic application might add and remove data storage VMs due to cost and performance requirements, varying the number of used VMs. Thus, additional information is needed in monitoring elastic applications, such as associating the logical application structure over the underlying virtual infrastructure. This in turn generates other problems, such as data aggregation from multiple VMs towards extracting the behavior of the application component running on top of those VMs, as different metrics may need to be aggregated differently (e.g., CPU usage could be averaged, network transfer might be summed up). === Stakeholder requirements === When deploying applications in cloud infrastructures (IaaS/PaaS), stakeholder requirements need to be considered in order to ensure that elastic behavior meets stakeholder needs. Traditionally, the optimal trade-off between cost and quality or performance is considered; however, for real world cloud users, requirements regarding elastic behavior are more complex and target multiple dimensions of elasticity (e.g., SYBL). === Multiple levels of control === Cloud applications vary in type and complexity, with multiple levels of artifacts deployed in layers. Controlling such structures must take into consideration a variety of issues. For multi-level control, control systems need to consider the impact lower level control has upon higher level ones, and vice versa (e.g., controlling virtual machines, web containers, or web services in the same time), as well as conflicts that may appear between various control strategies from various levels. Elastic strategies on in cloud computing can take advantage of control-theoretic methods (e.g., predictive control has been experimented in cloud computing scenarios by showing considerable advantages with respect to reactive methods). One approach to multi-level elastic clouc control is rSYBL.

Tokken

Tokken is a payment system and mobile app most known for being a legal and secure option for businesses transactions within the cannabis industry, because of its compliance with bank requirements. The startup company was created by Lamine Zarrad, a former regulator at the Office of the Comptroller of the Currency. == Operability == In order for a person to start using the app, they need to provide evidence, in the form of bioidentification data and mobile carrier records, that they can legally purchase weed. After they have been verified, customers can pay directly through the app at any dispensary that is using Tokken. Tokken turns credit card transactions into a digital token, which can be exchanged back for money that can later be deposited into a bank account. All transactions are logged publicly through a blockchain leger, making the process both anonymous and verified. === Banking services === Tokken has a "pay taxes" function which enables dispensaries to pay their taxes directly to the department.

Dyme (company)

Trebel (music app)

Israeli cybersecurity industry

Trebel (music app)

Elasticity (computing)

Tokken

Related Articles