In search of the best AI customer-support bot? An AI customer-support bot is software that uses machine learning to help you get more done — it turns a rough idea into a polished result in seconds. When choosing one, weigh output quality, pricing, export formats, and how well it fits the tools you already use. Whether you are a beginner or a pro, the right AI customer-support bot slots into your workflow and pays for itself fast. Below we compare features, pricing, and real output so you can choose with confidence.
Instance selection
Instance selection (or dataset reduction, or dataset condensation) is an important data pre-processing step that can be applied in many machine learning (or data mining) tasks. Approaches for instance selection can be applied for reducing the original dataset to a manageable volume, leading to a reduction of the computational resources that are necessary for performing the learning process. Algorithms of instance selection can also be applied for removing noisy instances, before applying learning algorithms. This step can improve the accuracy in classification problems. Algorithm for instance selection should identify a subset of the total available data to achieve the original purpose of the data mining (or machine learning) application as if the whole data had been used. Considering this, the optimal outcome of IS would be the minimum data subset that can accomplish the same task with no performance loss, in comparison with the performance achieved when the task is performed using the whole available data. Therefore, every instance selection strategy should deal with a trade-off between the reduction rate of the dataset and the classification quality. == Instance selection algorithms == The literature provides several different algorithms for instance selection. They can be distinguished from each other according to several different criteria. Considering this, instance selection algorithms can be grouped in two main classes, according to what instances they select: algorithms that preserve the instances at the boundaries of classes and algorithms that preserve the internal instances of the classes. Within the category of algorithms that select instances at the boundaries it is possible to cite DROP3, ICF and LSBo. On the other hand, within the category of algorithms that select internal instances, it is possible to mention ENN and LSSm. In general, algorithm such as ENN and LSSm are used for removing harmful (noisy) instances from the dataset. They do not reduce the data as the algorithms that select border instances, but they remove instances at the boundaries that have a negative impact on the data mining task. They can be used by other instance selection algorithms, as a filtering step. For example, the ENN algorithm is used by DROP3 as the first step, and the LSSm algorithm is used by LSBo. There is also another group of algorithms that adopt different selection criteria. For example, the algorithms LDIS, CDIS and XLDIS select the densest instances in a given arbitrary neighborhood. The selected instances can include both, border and internal instances. The LDIS and CDIS algorithms are very simple and select subsets that are very representative of the original dataset. Besides that, since they search by the representative instances in each class separately, they are faster (in terms of time complexity and effective running time) than other algorithms, such as DROP3 and ICF. Besides that, there is a third category of algorithms that, instead of selecting actual instances of the dataset, select prototypes (that can be synthetic instances). In this category it is possible to include PSSA, PSDSP and PSSP. The three algorithms adopt the notion of spatial partition (a hyperrectangle) for identifying similar instances and extract prototypes for each set of similar instances. In general, these approaches can also be modified for selecting actual instances of the datasets. The algorithm ISDSP adopts a similar approach for selecting actual instances (instead of prototypes).
Data processing unit
A data processing unit (DPU) is a programmable computer processor that tightly integrates a general-purpose CPU with network interface hardware. They are also occasionally called "IPUs" (infrastructure processing unit) or "SmartNICs". They can be used in place of traditional NICs to relieve the main CPU of complex networking responsibilities and other "infrastructural" duties; although their features vary, they may be used to perform encryption/decryption, serve as a firewall, handle TCP/IP, process HTTP requests, or even function as a hypervisor or storage controller. These devices can be attractive to cloud computing providers whose servers might otherwise spend a significant amount of CPU time on these tasks, cutting into the cycles they can provide to guests. They see use in other kinds of data center environments as well due to their improved power consumption efficiency for routine networking tasks compared to general-purpose CPUs.
Wayve
Wayve Technologies Ltd is a British autonomous driving technology company focused on developing self-driving vehicle systems through end-to-end deep learning. Founded in 2017 by researchers from the University of Cambridge, Wayve’s approach eschews detailed 3D maps and hand-coded rules, in favor of a self-learning “AI driver” that learns from camera data and driving experience. The London-headquartered startup has garnered significant attention and funding for its visually-based method. == History == Wayve was founded in Cambridge, England, on August 21, 2017, by Amar Shah and Alex Kendall, two machine learning PhD students at the University of Cambridge. Shah initially served as CEO while Kendall was CTO, and the pair set out to develop an unconventional self-driving car system using machine learning at every layer of the driving task. In May 2018, Wayve emerged from stealth mode with backing from early-stage investors. At this time the company had around 10 employees, and its advisory investors included Uber’s Chief Scientist, Zoubin Ghahramani, who shared Wayve’s vision of a learning-centric driving AI. In 2019, Wayve achieved a milestone by training a car to drive autonomously on public roads it had never seen before, using only cameras, a basic GPS map, and end-to-end deep learning control. The company moved its base to London and secured a $20 million Series A funding round in November 2019. This investment enabled Wayve to launch a pilot fleet of autonomous electric vehicles in central London for real-world testing. During these trials, Wayve’s cars (such as retrofitted Jaguar I-Pace SUVs) began navigating the complex, narrow streets of London to prove the system’s ability to adapt to challenging urban scenarios. In 2020, co-founder Amar Shah departed the company, and Alex Kendall assumed the role of CEO. The startup joined the Microsoft for Startups: Autonomous Driving program in 2020, leveraging Microsoft Azure’s cloud computing for training its machine learning models at scale. It also committed to testing exclusively on electric vehicles, and a goal to reduce carbon emissions. In 2021, Wayve entered pilot programs with major UK retailers. It launched a 12-month autonomous delivery trial with supermarket chain Asda, and received a £10 million ($13.6 million) investment from online grocer Ocado Group as part of a partnership to develop self-driving grocery delivery vans. Ocado’s backing gave Wayve access to a fleet of delivery vans for data collection and testing on busy London routes (with human safety drivers present) to train its AI in urban traffic. In 2022, after a successful Series B funding round, the company extended road testing beyond the UK to other regions, and, by 2023, in multiple countries. The company had begun operating in the United States and in continental Europe, in preparation for larger commercial deployments. In 2023, Wayve announced a collaboration with Nissan to integrate Wayve’s AI-driven software into its ProPilot ADAS system, slated to launch in fiscal year 2027. Wayve received strategic investment from Uber, in 2024, to jointly develop autonomous ride-hailing services. The two companies plan to trial a fully driverless robotaxi service in London, supported by a UK government program to accelerate commercial self-driving pilots to as early as 2026. To demonstrate the scalability of its technology, Wayve conducted an “AI-500” roadshow project, driving in dozens of cities across Asia, Europe, and North America using the same AI model. By mid-2025, it had completed autonomous driving demos in 90 cities without prior HD mapping. In April 2025, Wayve opened its first Asian research hub in Japan, with investment by SoftBank, to improve its model’s generalization using local driving data. That year, the company conducted driving tests in over 500 cities in Europe, North America and Japan without city-specific programming. In February 2026, Nissan, Uber and Wayve announced their collaboration on robotaxi development, with the aim of launching a pilot programme in Tokyo by late 2026. Wayve also formed a strategic alliance with Mercedes-Benz and Stellantis on personal vehicle and robotaxi applications. == Financing and investors == Wayve has been backed by a mix of venture capital (VC) firms, corporate investors, and individuals. Its initial seed funding came from funds such as Compound (NYC) and Firstminute Capital (London), as well as Cambridge-based angel investors, in 2018. Academic Pieter Abbeel and Uber’s chief scientist, Zoubin Ghahramani, were early backers. In November 2019, Wayve raised a $20 million Series A led by Eclipse Ventures, with participation from Balderton Capital and other prior investors. The Series A financing was used to fund the company’s first autonomous trials in London, and marked the first time a European self-driving car startup had secured a U.S. VC as lead investor. In October 2021, Ocado Group invested £10 million (approximately $13.6 million) in Wayve as a strategic partner in autonomous grocery delivery. This brought Wayve’s total funding to around $60 million at that time. The Series B round followed in January 2022, when Wayve announced $200 million in new funding led by Eclipse Ventures, with D1 Capital Partners, Moore Strategic Ventures, and Linse Capital. Balderton, Microsoft and Virgin Group joined as strategic backers. Baillie Gifford and Compound also participated; Ocado increased its stake as a strategic investor; and Meta AI head Yann LeCun and Richard Branson also became investors. Wayve’s Series C in May 2024 closed a $1.05 billion, led by Japan’s SoftBank Group. The funding round was the largest-ever for a UK AI company, and included new investor Nvidia, and returning investors Microsoft and Eclipse Ventures, among others. Uber also joined as a stratgic partner and a stakeholder. The Series C round increased Wayve’s total funding raised to about $1.3 billion to date from investors including SoftBank, Microsoft and Nvidia, and lifted Wayve’s valuation into “unicorn” status. In February 2026, Wayve announced a $1.2 billion Series D funding round; later that month, the company reported that $1.5 billion had been raised from, primarily, Mercedes-Benz, Stellantis, Nissan, and existing backers Uber, Microsoft and Nvidia, increasing Wayve's overall valuation to $8.6 billion. == Technology == Wayve’s self-driving approach centers on end-to-end deep learning and a vision-based AI system. Unlike conventional autonomous vehicles that depend on high-definition maps, hand-coded rules, and arrays of expensive lidar sensors, Wayve’s platform learns to drive predominantly using camera data and machine learning algorithms. The company refers to its AI-driven driving software as an “Embodied AI” or AI Driver, emphasizing that the system learns from experience (both real and simulated) to handle complex or novel situations rather than following pre-programmed instructions, not unlike Tesla's approach. The Wayve hardware-agnostic autonomy stack consists of a suite of video cameras, with basic automotive sensors, mounted on the vehicle, and paired with onboard compute units that are powered by GPUs to run the AI models. This vision-only philosophy is similar to Tesla’s Autopilot/FSDB model, but Wayve’s solution is vehicle-agnostic and mapless. Wayve’s strategy is to provide its driving AI as an OEM-ready platform; it plans to license or embed its technology into vehicles made by established automakers rather than build its own cars. Wayve’s development vehicles currently use Nvidia’s Orin system-on-chip as the onboard computer for running the AI model, but CEO Kendall has noted that the software can run on “whatever GPU [an automaker] already has in their vehicles” Wayve has built a cloud infrastructure, largely on Microsoft Azure, to process petabytes of this data, and uses simulation tools (known internally as the “Wayve Infinity” simulator) to synthetically generate and practice rare or dangerous scenarios for the AI to learn from. == Corporate affairs == Wayve is a privately held company headquartered in London, England, with its primary research and development office in the Kings Cross area of London. The company was initially incorporated as Wayve Technologies Ltd in the UK. Wayve has also established a presence in the U.S., in Silicon Valley); in Canada, with a research hub in Vancouver; in Yokohama, Japan; in Leonberg, Germany; and in Herzliya, Israel. The Leadership team includes research scientists and engineers with backgrounds in computer vision, robotics, and automotive systems. President Erez Dagan was hired in 2024, following two decades at Mobileye; chief scientist Jamie Shotton is formerly of Microsoft Research; CEO Alex Kendall, originally from New Zealand with a PhD in computer vision from Cambridge, took over as CEO in 2020 after the departure of his co-founder Amar Shah.
Type-1 OWA operators
Type-1 OWA operators are a set of aggregation operators that generalise the Yager's OWA (ordered weighted averaging) operators in the interest of aggregating fuzzy sets rather than crisp values in soft decision making and data mining. These operators provide a mathematical technique for directly aggregating uncertain information with uncertain weights via OWA mechanism in soft decision making and data mining, where these uncertain objects are modelled by fuzzy sets. The two definitions for type-1 OWA operators are based on Zadeh's Extension Principle and α {\displaystyle \alpha } -cuts of fuzzy sets. The two definitions lead to equivalent results. == Definitions == === Definition 1 === Let F ( X ) {\displaystyle F(X)} be the set of fuzzy sets with domain of discourse X {\displaystyle X} , a type-1 OWA operator is defined as follows: Given n linguistic weights { W i } i = 1 n {\displaystyle \left\{{W^{i}}\right\}_{i=1}^{n}} in the form of fuzzy sets defined on the domain of discourse U = [ 0 , 1 ] {\displaystyle U=[0,1]} , a type-1 OWA operator is a mapping, Φ {\displaystyle \Phi } , Φ : F ( X ) × ⋯ × F ( X ) ⟶ F ( X ) {\displaystyle \Phi \colon F(X)\times \cdots \times F(X)\longrightarrow F(X)} ( A 1 , ⋯ , A n ) ↦ Y {\displaystyle (A^{1},\cdots ,A^{n})\mapsto Y} such that μ Y ( y ) = sup ∑ k = 1 n w ¯ i a σ ( i ) = y ( μ W 1 ( w 1 ) ∧ ⋯ ∧ μ W n ( w n ) ∧ μ A 1 ( a 1 ) ∧ ⋯ ∧ μ A n ( a n ) ) {\displaystyle \mu _{Y}(y)=\displaystyle \sup _{\displaystyle \sum _{k=1}^{n}{\bar {w}}_{i}a_{\sigma (i)}=y}\left({\begin{array}{{1}l}\mu _{W^{1}}(w_{1})\wedge \cdots \wedge \mu _{W^{n}}(w_{n})\wedge \mu _{A^{1}}(a_{1})\wedge \cdots \wedge \mu _{A^{n}}(a_{n})\end{array}}\right)} where w ¯ i = w i ∑ i = 1 n w i {\displaystyle {\bar {w}}_{i}={\frac {w_{i}}{\sum _{i=1}^{n}{w_{i}}}}} , and σ : { 1 , ⋯ , n } ⟶ { 1 , ⋯ , n } {\displaystyle \sigma \colon \{1,\cdots ,n\}\longrightarrow \{1,\cdots ,n\}} is a permutation function such that a σ ( i ) ≥ a σ ( i + 1 ) , ∀ i = 1 , ⋯ , n − 1 {\displaystyle a_{\sigma (i)}\geq a_{\sigma (i+1)},\ \forall i=1,\cdots ,n-1} , i.e., a σ ( i ) {\displaystyle a_{\sigma (i)}} is the i {\displaystyle i} th highest element in the set { a 1 , ⋯ , a n } {\displaystyle \left\{{a_{1},\cdots ,a_{n}}\right\}} . === Definition 2 === Using the alpha-cuts of fuzzy sets: Given the n linguistic weights { W i } i = 1 n {\displaystyle \left\{{W^{i}}\right\}_{i=1}^{n}} in the form of fuzzy sets defined on the domain of discourse U = [ 0 , 1 ] {\displaystyle U=[0,\;\;1]} , then for each α ∈ [ 0 , 1 ] {\displaystyle \alpha \in [0,\;1]} , an α {\displaystyle \alpha } -level type-1 OWA operator with α {\displaystyle \alpha } -level sets { W α i } i = 1 n {\displaystyle \left\{{W_{\alpha }^{i}}\right\}_{i=1}^{n}} to aggregate the α {\displaystyle \alpha } -cuts of fuzzy sets { A i } i = 1 n {\displaystyle \left\{{A^{i}}\right\}_{i=1}^{n}} is: Φ α ( A α 1 , … , A α n ) = { ∑ i = 1 n w i a σ ( i ) ∑ i = 1 n w i | w i ∈ W α i , a i ∈ A α i , i = 1 , … , n } {\displaystyle \Phi _{\alpha }\left({A_{\alpha }^{1},\ldots ,A_{\alpha }^{n}}\right)=\left\{{{\frac {\sum \limits _{i=1}^{n}{w_{i}a_{\sigma (i)}}}{\sum \limits _{i=1}^{n}{w_{i}}}}\left|{w_{i}\in W_{\alpha }^{i},\;a_{i}}\right.\in A_{\alpha }^{i},\;i=1,\ldots ,n}\right\}} where W α i = { w | μ W i ( w ) ≥ α } , A α i = { x | μ A i ( x ) ≥ α } {\displaystyle W_{\alpha }^{i}=\{w|\mu _{W_{i}}(w)\geq \alpha \},A_{\alpha }^{i}=\{x|\mu _{A_{i}}(x)\geq \alpha \}} , and σ : { 1 , ⋯ , n } → { 1 , ⋯ , n } {\displaystyle \sigma :\{\;1,\cdots ,n\;\}\to \{\;1,\cdots ,n\;\}} is a permutation function such that a σ ( i ) ≥ a σ ( i + 1 ) , ∀ i = 1 , ⋯ , n − 1 {\displaystyle a_{\sigma (i)}\geq a_{\sigma (i+1)},\;\forall \;i=1,\cdots ,n-1} , i.e., a σ ( i ) {\displaystyle a_{\sigma (i)}} is the i {\displaystyle i} th largest element in the set { a 1 , ⋯ , a n } {\displaystyle \left\{{a_{1},\cdots ,a_{n}}\right\}} . == Representation theorem of Type-1 OWA operators == Given the n linguistic weights { W i } i = 1 n {\displaystyle \left\{{W^{i}}\right\}_{i=1}^{n}} in the form of fuzzy sets defined on the domain of discourse U = [ 0 , 1 ] {\displaystyle U=[0,\;\;1]} , and the fuzzy sets A 1 , ⋯ , A n {\displaystyle A^{1},\cdots ,A^{n}} , then we have that Y = G {\displaystyle Y=G} where Y {\displaystyle Y} is the aggregation result obtained by Definition 1, and G {\displaystyle G} is the result obtained by in Definition 2. == Programming problems for Type-1 OWA operators == According to the Representation Theorem of Type-1 OWA Operators, a general type-1 OWA operator can be decomposed into a series of α {\displaystyle \alpha } -level type-1 OWA operators. In practice, this series of α {\displaystyle \alpha } -level type-1 OWA operators is used to construct the resulting aggregation fuzzy set. So we only need to compute the left end-points and right end-points of the intervals Φ α ( A α 1 , ⋯ , A α n ) {\displaystyle \Phi _{\alpha }\left({A_{\alpha }^{1},\cdots ,A_{\alpha }^{n}}\right)} . Then, the resulting aggregation fuzzy set is constructed with the membership function as follows: μ G ( x ) = ⋁ α : x ∈ Φ α ( A α 1 , ⋯ , A α n ) α α {\displaystyle \mu _{G}(x)=\operatorname {\bigvee } \limits _{\alpha :x\in \Phi _{\alpha }\left({A_{\alpha }^{1},\cdots ,A_{\alpha }^{n}}\right)_{\alpha }}\alpha } For the left end-points, we need to solve the following programming problem: Φ α ( A α 1 , ⋯ , A α n ) − = min W α − i ≤ w i ≤ W α + i A α − i ≤ a i ≤ A α + i ∑ i = 1 n w i a σ ( i ) / ∑ i = 1 n w i {\displaystyle \Phi _{\alpha }\left({A_{\alpha }^{1},\cdots ,A_{\alpha }^{n}}\right)_{-}=\operatorname {\min } \limits _{\begin{array}{l}W_{\alpha -}^{i}\leq w_{i}\leq W_{\alpha +}^{i}A_{\alpha -}^{i}\leq a_{i}\leq A_{\alpha +}^{i}\end{array}}\sum \limits _{i=1}^{n}{w_{i}a_{\sigma (i)}/\sum \limits _{i=1}^{n}{w_{i}}}} while for the right end-points, we need to solve the following programming problem: Φ α ( A α 1 , ⋯ , A α n ) + = max W α − i ≤ w i ≤ W α + i A α − i ≤ a i ≤ A α + i ∑ i = 1 n w i a σ ( i ) / ∑ i = 1 n w i {\displaystyle \Phi _{\alpha }\left({A_{\alpha }^{1},\cdots ,A_{\alpha }^{n}}\right)_{+}=\operatorname {\max } \limits _{\begin{array}{l}W_{\alpha -}^{i}\leq w_{i}\leq W_{\alpha +}^{i}A_{\alpha -}^{i}\leq a_{i}\leq A_{\alpha +}^{i}\end{array}}\sum \limits _{i=1}^{n}{w_{i}a_{\sigma (i)}/\sum \limits _{i=1}^{n}{w_{i}}}} A fast method has been presented to solve two programming problem so that the type-1 OWA aggregation operation can be performed efficiently, for details, please see the paper. == Alpha-level approach to Type-1 OWA operation == Three-step process: Step 1—To set up the α {\displaystyle \alpha } - level resolution in [0, 1]. Step 2—For each α ∈ [ 0 , 1 ] {\displaystyle \alpha \in [0,1]} , Step 2.1—To calculate ρ α + i 0 ∗ {\displaystyle \rho _{\alpha +}^{i_{0}^{\ast }}} Let i 0 = 1 {\displaystyle i_{0}=1} ; If ρ α + i 0 ≥ A α + σ ( i 0 ) {\displaystyle \rho _{\alpha +}^{i_{0}}\geq A_{\alpha +}^{\sigma (i_{0})}} , stop, ρ α + i 0 {\displaystyle \rho _{\alpha +}^{i_{0}}} is the solution; otherwise go to Step 2.1-3. i 0 ← i 0 + 1 {\displaystyle i_{0}\leftarrow i_{0}+1} , go to Step 2.1-2. Step 2.2 To calculate ρ α − i 0 ∗ {\displaystyle \rho _{\alpha -}^{i_{0}^{\ast }}} Let i 0 = 1 {\displaystyle i_{0}=1} ; If ρ α − i 0 ≥ A α − σ ( i 0 ) {\displaystyle \rho _{\alpha -}^{i_{0}}\geq A_{\alpha -}^{\sigma (i_{0})}} , stop, ρ α − i 0 {\displaystyle \rho _{\alpha -}^{i_{0}}} is the solution; otherwise go to Step 2.2-3. i 0 ← i 0 + 1 {\displaystyle i_{0}\leftarrow i_{0}+1} , go to step Step 2.2-2. Step 3—To construct the aggregation resulting fuzzy set G {\displaystyle G} based on all the available intervals [ ρ α − i 0 ∗ , ρ α + i 0 ∗ ] {\displaystyle \left[{\rho _{\alpha -}^{i_{0}^{\ast }},\;\rho _{\alpha +}^{i_{0}^{\ast }}}\right]} : μ G ( x ) = ⋁ α : x ∈ [ ρ α − i 0 ∗ , ρ α + i 0 ∗ ] α {\displaystyle \mu _{G}(x)=\operatorname {\bigvee } \limits _{\alpha :x\in \left[{\rho _{\alpha -}^{i_{0}^{\ast }},\;\rho _{\alpha +}^{i_{0}^{\ast }}}\right]}\alpha } == Some Examples == The type-1 OWA operator with the weights shown in the top figure is used to aggregate the fuzzy sets (solide lines) in the bottom figure, and the dashed line is the aggregation result. == Special cases == Any OWA operators, like maximum, minimum, mean operators; Join operators of (type-1) fuzzy sets, i.e., fuzzy maximum operators; Meet operators of (type-1) fuzzy sets, i.e., fuzzy minimum operators; Join-like operators of (type-1) fuzzy sets; Meet-like operators of (type-1) fuzzy sets. == Generalizations == Type-2 OWA operators have been suggested to aggregate the type-2 fuzzy sets for soft decision making. == Applications == Type-1 OWA operators have been applied to different domains for soft decision making. Improved efficiency of computing approach ; Type reduction of type-2 fuzzy sets ; Group decision making ; Credit risk evaluation ; Information fusion ; Linguistic expressions and symbolic translation ; Sentiment analysis ; Ro
Network Abstraction Layer
The Network Abstraction Layer (NAL) is a part of the H.264/AVC and HEVC video coding standards. The main goal of the NAL is the provision of a "network-friendly" video representation addressing "conversational" (video telephony) and "non conversational" (storage, broadcast, or streaming) applications. NAL has achieved a significant improvement in application flexibility relative to prior video coding standards. == Introduction == An increasing number of services and growing popularity of high definition TV are creating greater needs for higher coding efficiency. Moreover, other transmission media such as cable modem, xDSL, or UMTS offer much lower data rates than broadcast channels, and enhanced coding efficiency can enable the transmission of more video channels or higher quality video representations within existing digital transmission capacities. Video coding for telecommunication applications has diversified from ISDN and T1/E1 service to embrace PSTN, mobile wireless networks, and LAN/Internet network delivery. Throughout this evolution, continued efforts have been made to maximize coding efficiency while dealing with the diversification of network types and their characteristic formatting and loss/error robustness requirements. The H.264/AVC and HEVC standards are designed for technical solutions including areas like broadcasting (over cable, satellite, cable modem, DSL, terrestrial, etc.) interactive or serial storage on optical and magnetic devices, conversational services, video-on-demand or multimedia streaming, multimedia messaging services, etc. Moreover, new applications may be deployed over existing and future networks. This raises the question about how to handle this variety of applications and networks. To address this need for flexibility and customizability, the design covers a NAL that formats the Video Coding Layer (VCL) representation of the video and provides header information in a manner appropriate for conveyance by a variety of transport layers or storage media. The NAL is designed in order to provide "network friendliness" to enable simple and effective customization of the use of VCL for a broad variety of systems. The NAL facilitates the ability to map VCL data to transport layers such as: RTP/IP for any kind of real-time wire-line and wireless Internet services. File formats, e.g., ISO MP4 for storage and MMS. H.32X for wireline and wireless conversational services. MPEG-2 systems for broadcasting services, etc. The full degree of customization of the video content to fit the needs of each particular application is outside the scope of the video coding standardization effort, but the design of the NAL anticipates a variety of such mappings. Some key concepts of the NAL are NAL units, byte stream, and packet formats uses of NAL units, parameter sets, and access units. A short description of these concepts is given below. == NAL units == The coded video data is organized into NAL units, each of which is effectively a packet that contains an integer number of bytes. The first byte of each H.264/AVC NAL unit is a header byte that contains an indication of the type of data in the NAL unit. For HEVC the header was extended to two bytes. All the remaining bytes contain payload data of the type indicated by the header. The NAL unit structure definition specifies a generic format for use in both packet-oriented and bitstream-oriented transport systems, and a series of NAL units generated by an encoder is referred to as a NAL unit stream. == NAL Units in Byte-Stream Format Use == Some systems require delivery of the entire or partial NAL unit stream as an ordered stream of bytes or bits within which the locations of NAL unit boundaries need to be identifiable from patterns within the coded data itself. For use in such systems, the H.264/AVC and HEVC specifications define a byte stream format. In the byte stream format, each NAL unit is prefixed by a specific pattern of three bytes called a start code prefix. The boundaries of the NAL unit can then be identified by searching the coded data for the unique start code prefix pattern. The use of emulation prevention bytes guarantees that start code prefixes are unique identifiers of the start of a new NAL unit. A small amount of additional data (one byte per video picture) is also added to allow decoders that operate in systems that provide streams of bits without alignment to byte boundaries to recover the necessary alignment from the data in the stream. Additional data can also be inserted in the byte stream format that allows expansion of the amount of data to be sent and can aid in achieving more rapid byte alignment recovery, if desired. == NAL Units in Packet-Transport System Use == In other systems (e.g., IP/RTP systems), the coded data is carried in packets that are framed by the system transport protocol, and identification of the boundaries of NAL units within the packets can be established without use of start code prefix patterns. In such systems, the inclusion of start code prefixes in the data would be a waste of data carrying capacity, so instead the NAL units can be carried in data packets without start code prefixes. == VCL and Non-VCL NAL Units == NAL units are classified into VCL and non-VCL NAL units. VCL NAL units contain the data that represents the values of the samples in the video pictures. Non-VCL NAL units contain any associated additional information such as parameter sets (important header data that can apply to a large number of VCL NAL units) and supplemental enhancement information (timing information and other supplemental data that may enhance usability of the decoded video signal but are not necessary for decoding the values of the samples in the video pictures). == Parameter Sets == A parameter set contains shared configuration data that is carried in non-VCL NAL units. Parameter sets are typically reused when decoding many coded pictures within a video sequence. Each VCL NAL unit references a picture parameter set (PPS), which in turn references a sequence parameter set (SPS). There are two types of parameter sets: Sequence parameter set (SPS), which specifies mostly constant configuration such as resolution, bit depth, or chroma format. (For a concrete implementation, see FFmpeg's SPS struct.) Picture parameter set (PPS), which applies on top of an SPS, and specifies configuration such as QP offsets. (For a concrete implementation, see FFmpeg's PPS struct.) The sequence and picture parameter-set mechanism decouples the transmission of infrequently changing information from the transmission of coded representations of the values of the samples in the video pictures. Each VCL NAL unit contains an identifier that refers to the content of the relevant picture parameter set and each picture parameter set contains an identifier that refers to the content of the relevant sequence parameter set. In this manner, a small amount of data (the identifier) can be used to refer to a larger amount of information (the parameter set) without repeating that information within each VCL NAL unit. Sequence and picture parameter sets can be sent well ahead of the VCL NAL units that they apply to, and can be repeated to provide robustness against data loss. In some applications, parameter sets may be sent within the channel that carries the VCL NAL units (termed "in-band" transmission). In other applications, it can be advantageous to convey the parameter sets "out-of-band" using a more reliable transport mechanism than the video channel itself. == Access Units == A set of NAL units in a specified form is referred to as an access unit. The decoding of each access unit results in one decoded picture. Each access unit contains a set of VCL NAL units that together compose a primary coded picture. It may also be prefixed with an access unit delimiter to aid in locating the start of the access unit. Some supplemental enhancement information containing data such as picture timing information may also precede the primary coded picture. The primary coded picture consists of a set of VCL NAL units consisting of slices or slice data partitions that represent the samples of the video picture. Following the primary coded picture may be some additional VCL NAL units that contain redundant representations of areas of the same video picture. These are referred to as redundant coded pictures, and are available for use by a decoder in recovering from loss or corruption of the data in the primary coded pictures. Decoders are not required to decode redundant coded pictures if they are present. Finally, if the coded picture is the last picture of a coded video sequence (a sequence of pictures that is independently decodable and uses only one sequence parameter set), an end of sequence NAL unit may be present to indicate the end of the sequence; and if the coded picture is the last coded picture in the entire NAL unit stream, an end of stream NAL unit may be present to
Stable Diffusion
Stable Diffusion is a deep learning, text-to-image model released in 2022 based on diffusion techniques. The generative artificial intelligence technology is the premier product of Stability AI and is considered to be a part of the ongoing AI boom. It is primarily used to generate detailed images conditioned on text descriptions, though it can also be applied to other tasks such as inpainting, outpainting, and generating image-to-image translations guided by a text prompt. Its development involved researchers from the CompVis Group at LMU Munich and Runway with a computational donation from Stability and training data from non-profit organizations. Stable Diffusion is a latent diffusion model, a kind of deep generative artificial neural network. Its code and model weights have been released publicly, and an optimized version can run on most consumer hardware equipped with a modest GPU with as little as 2.4 GB VRAM. This marked a departure from previous proprietary text-to-image models such as DALL-E and Midjourney which were accessible only via cloud services. == Development == Stable Diffusion originated from a project called Latent Diffusion, developed in Germany by researchers at LMU Munich in Munich and Heidelberg University. Four of the original 5 authors (Robin Rombach, Andreas Blattmann, Patrick Esser and Dominik Lorenz) later joined Stability AI and released subsequent versions of Stable Diffusion. The technical license for the model was released by the CompVis group at LMU Munich. Development was led by Patrick Esser of Runway and Robin Rombach of CompVis, who were among the researchers who had earlier invented the latent diffusion model architecture used by Stable Diffusion. Stability AI also credited EleutherAI and LAION (a German nonprofit which assembled the dataset on which Stable Diffusion was trained) as supporters of the project. == Technology == === Architecture === Diffusion models, introduced in 2015, are trained with the objective of removing successive applications of Gaussian noise on training images, which can be thought of as a sequence of denoising autoencoders. The name diffusion is from the thermodynamic diffusion, since they were first developed with inspiration from thermodynamics. Models in Stable Diffusion series before SD 3 all used a variant of diffusion models, called latent diffusion model (LDM), developed in 2021 by the CompVis (Computer Vision & Learning) group at LMU Munich. Stable Diffusion consists of 3 parts: the variational autoencoder (VAE), U-Net, and an optional text encoder. The VAE encoder compresses the image from pixel space to a smaller dimensional latent space, capturing a more fundamental semantic meaning of the image. Gaussian noise is iteratively applied to the compressed latent representation during forward diffusion. The U-Net block, composed of a ResNet backbone, denoises the output from forward diffusion backwards to obtain a latent representation. Finally, the VAE decoder generates the final image by converting the representation back into pixel space. The denoising step can be flexibly conditioned on a string of text, an image, or another modality. The encoded conditioning data is exposed to denoising U-Nets via a cross-attention mechanism. For conditioning on text, the fixed, pretrained CLIP ViT-L/14 text encoder is used to transform text prompts to an embedding space. Researchers point to increased computational efficiency for training and generation as an advantage of LDMs. With 860 million parameters in the U-Net and 123 million in the text encoder, Stable Diffusion is considered relatively lightweight by 2022 standards, and unlike other diffusion models, it can run on consumer GPUs, and even CPU-only if using the OpenVINO version of Stable Diffusion. ==== SD XL ==== The XL version uses the same LDM architecture as previous versions, except larger: larger UNet backbone, larger cross-attention context, two text encoders instead of one, and trained on multiple aspect ratios (not just the square aspect ratio like previous versions). The SD XL Refiner, released at the same time, has the same architecture as SD XL, but it was trained for adding fine details to preexisting images via text-conditional img2img. ==== SD 3.0 ==== The 3.0 version completely changes the backbone. Not a UNet, but a Rectified Flow Transformer, which implements the rectified flow method with a Transformer. The Transformer architecture used for SD 3.0 has three "tracks", for original text encoding, transformed text encoding, and image encoding (in latent space). The transformed text encoding and image encoding are mixed during each transformer block. The architecture is named "multimodal diffusion transformer (MMDiT), where the "multimodal" means that it mixes text and image encodings inside its operations. This differs from previous versions of DiT, where the text encoding affects the image encoding, but not vice versa. === Training data === Stable Diffusion was trained on pairs of images and captions taken from LAION-5B, a publicly available dataset derived from Common Crawl data scraped from the web, where 5 billion image-text pairs were classified based on language and filtered into separate datasets by resolution, a predicted likelihood of containing a watermark, and predicted "aesthetic" score (e.g. subjective visual quality). The dataset was created by LAION, a German non-profit which receives funding from Stability AI. The Stable Diffusion model was trained on three subsets of LAION-5B: laion2B-en, laion-high-resolution, and laion-aesthetics v2 5+. A third-party analysis of the model's training data identified that out of a smaller subset of 12 million images taken from the original wider dataset used, approximately 47% of the sample size of images came from 100 different domains, with Pinterest taking up 8.5% of the subset, followed by websites such as WordPress, Blogspot, Flickr, DeviantArt and Wikimedia Commons. An investigation by Bayerischer Rundfunk showed that LAION's datasets, hosted on Hugging Face, contain large amounts of private and sensitive data. === Training procedures === The model was initially trained on the laion2B-en and laion-high-resolution subsets, with the last few rounds of training done on LAION-Aesthetics v2 5+, a subset of 600 million captioned images which the LAION-Aesthetics Predictor V2 predicted that humans would, on average, give a score of at least 5 out of 10 when asked to rate how much they liked them. The LAION-Aesthetics v2 5+ subset also excluded low-resolution images and images which LAION-5B-WatermarkDetection identified as carrying a watermark with greater than 80% probability. Final rounds of training additionally dropped 10% of text conditioning to improve Classifier-Free Diffusion Guidance. The model was trained using 256 Nvidia A100 GPUs on Amazon Web Services for a total of 150,000 GPU-hours, at a cost of $600,000. === Limitations === Stable Diffusion has issues with degradation and inaccuracies in certain scenarios. Initial releases of the model were trained on a dataset that consists of 512×512 resolution images, meaning that the quality of generated images noticeably degrades when user specifications deviate from its "expected" 512×512 resolution; the version 2.0 update of the Stable Diffusion model later introduced the ability to natively generate images at 768×768 resolution. Another challenge is in generating human limbs due to poor data quality of limbs in the LAION database. The model is insufficiently trained to replicate human limbs and faces due to the lack of representative features in the database, and prompting the model to generate images of such type can confound the model. In addition to human limbs, Stable Diffusion is unable to generate legible ambigrams and some other forms of text and typography. Stable Diffusion XL (SDXL) version 1.0, released in July 2023, introduced native 1024x1024 resolution and improved generation for limbs and text. Accessibility for individual developers can also be a problem. In order to customize the model for new use cases that are not included in the dataset, such as generating anime characters ("waifu diffusion"), new data and further training are required. Fine-tuned adaptations of Stable Diffusion created through additional retraining have been used for a variety of different use-cases, from medical imaging to algorithmically generated music. However, this fine-tuning process is sensitive to the quality of new data; low resolution images or different resolutions from the original data can not only fail to learn the new task but degrade the overall performance of the model. Even when the model is additionally trained on high quality images, it is difficult for individuals to run models in consumer electronics. For example, the training process for waifu-diffusion requires a minimum 30 GB of VRAM, which exceeds the usual resource provided in such consumer GPUs as Nvidia's GeForce 30 series, w